Andrew Chen Archives

2012 has been good to startups.
It’s never been easier to raise seed funding, and there’s warnings that we’re in the midst of a “seed stage bubble.” Whether you think it’s a bubble or a boom things are good- you have to ask, how long will the party go on?

My theory is, we’re currently in a golden age for early stage startups, and the early stage market will stay hot for at least the next 3-5 years.

Here’s why the good times will continue
My reasoning looks something like this:

1. Right now, startups with strong teams can easily raise seed funding ($200-$1.5M or so)
2. They can easily raise seed money because there’s a lot of willing investors in the ecosystem. VCs are seeding deals without any price sensitivity, and a lot of angels seeing exits even when the teams fail
3. Angels are willing to invest because they have downside protection due to acquihires. They can invest $50-200k per deal and in the event of a startup failure, they get their money back (and sometimes even get marked up to a profit!). If the startups succeed, they have tremendous upside.
4. The downside protection is driven by acquihires from companies like Twitter, Facebook, Groupon, and others which are paying $1M-$3M per engineer. This makes sense to them because there are multiple billion-dollar markets at play.
5. And ironically, because this whole system exists, the engineers at great startups feel like they can splinter off and start their own thing, which feeds into the whole thing

Given that team acquisitions provide downside protection while the hits drive the real returns, it’s hard for investors behind top teams to lose money. So the question is, when will the downside protection, in the form of acquihires, disappear?

Mobile as the driver
IMHO, the answer to that key question is, I think we’re another 3-5 years because of one key thing that’s driving all of it: iPhone. (And Android, and the rest of the smartphone industry).

It’s going to take 3-5 years for the mobile market to sort itself out. As long as smartphones are still progressing from their current 100s of millions to the final 3B active users number, every company will be investing in this new platform, and they’ll keep buying as to not get left behind. Otherwise, they’ll be left on a previous platform as a new competitor emerges that’s mobile centric, and smokes them.

There’s a whole host of companies in the Bay Area, in Asia, and around the world that are investing heavily on mobile. They’ll buy any team they can get their hands on.

So they’ll keep acquihiring talent, supporting the whole thing, until the mobile market is set.

Whether you think this is a good thing or a bad thing, IMHO the mobile wave is so huge that it has the ability to power the early stage investing marketplace for years. Agree? Disagree? Tell me in the comments.

Stop the madness
The long cycle times for developing mobile apps have led to startup failures that look more like 1999 – it’s like we’ve forgotten all the agile and rapid iteration stuff that we learned over the last 10 years. Stop the madness!

Today, seed stage startups can now get funded, release 1 or 2 versions of their app spread over 9 months, and then fail without making a peep. We learned the benefits of how to iterate fast on the web, and we can do better on mobile too.

How things worked in 1999
How’d we get here? Back in 1999, we did a similar thing:

• Raise millions in funding with an idea and impressive founders
• Spend 9 months building up a product
• Launch with much PR fanfare
• Fail to hit product/market fit
• Relaunch with version 2.0, 6 months later
• Repeat until you run out of money

This was Pets.com, Kozmo, and so on. Maybe you’d fire your VP Marketing in the process too, out of frustration.

Between 2002-2009, we learned a lot of great ways to work quickly, deploy code a few times a week, and get very iterative about proving out your product.

How things work today
Then, with the arrival of the big smartphone platforms, we’ve reverted. It looks like 1999 but instead of launching, we submit into the iOS App Store.

It looks like this instead:

• Raise funding with an idea and impressive founders
• Spend 6 months building up a product
• Submit to the app store and launch with much PR fanfare
• Fail to hit product/market fit
• Relaunch with version 2.0, 6 months later
• Add Facebook Open Graph
• Try buying installs with Tapjoy, FreeAppADay, etc.
• Repeat until you run out of money

Not much different, unfortunately.

The platform reflects its master
We’ve gotten here because the App Store reflects Apple’s DNA of great products plus big launches. They are a 1980s hardware company that’s mastered that strategy, and when developers build on their platform, they have no choice but to emulate the approach as well.

Worse yet, it lets people indulge in a little fantasy that they too are Steve Jobs, and once they launch a polished product after months of work, they’ll be a huge success too. The emphasis on highly polished design for mobile products reverts us back to a waterfall development mentality.

Don’t burn 1/2 of your funding to get to a v1
Startups today have a super high bar for initial quality in their version 1. They also want to make a big press release about it, to drive traffic, since there’s really no other approach to succeed in mobile. And so we see startups burn 1/3 to 1/2 of their seed round before they release anything, it becomes really dangerous when the initial launch inevitably fails to catch fire. Then the rest of the funding isn’t enough to do a substantive update.

What can we do?
How can we stop the madness? What can do we do to combine the agility we learned in the past decade with the requirements of the App Store?

If we can answer this question, we’ll be much better off as an industry.

One of the most compelling organizational things I’ve read about lately is Square’s practice of referring to their product team as Product Editors and the product editorial team, rather than the traditional “Product Management” title. Wanted to share some quick thoughts below about it.

Product managers: One of the toughest and worst defined jobs in tech
The role of “product manager” “program manager” “project manager” is one of the toughest, and worst defined jobs in tech. And it often doesn’t lead to good products. The various PM roles often have no direct reports, but you have the responsibility of getting products out the door. It often becomes a detail-oriented role that are as much about hitting milestones and schedules as much as delivering a great product experience.

Thus PMs sometimes end up in the world of Gantt charts, 100-page spec documents, and spreadsheets rather than thinking about products. Now, all the scheduling and management tasks matter, but it’s too easy for PMs to lead with them rather than leading with products first.

Bad ideas are often good ideas that don’t fit
In the context of literature, books, and newspapers, it’s the job of the editor to pick the good stuff and weave it into a coherent story. You remove the bad stuff, but “bad” can mean it’s a good idea but just doesn’t fit into the story. It’s a compelling and important distinction for consumer internet.

Cohesion and consistency is difficult. When you have an organization with lots of very smart people all with their own good ideas, it’s difficult to decide which path to take. So often, products are compromised as the product “manager” doesn’t feel the responsibility to build up that cohesion as an ends in itself, and instead just tries to do as much as possible with the product given some set timeframe. Focus, people!

Jack Dorsey in his own words
In a recent talk at Stanford, Jack Dorsey describes his idea of editors:

“I’ve often spoken to the editorial nature of what I think my job is, I think I’m just an editor, and I think every CEO is an editor. I think every leader in any company is an editor. Taking all of these ideas and editing them down to one cohesive story, and in my case my job is to edit the team, so we have a great team that can produce the great work and that means bringing people on and in some cases having to let people go. That means editing the support for the company, which means having money in the bank, or making money, and that means editing what the vision and the communication of the company is, so that’s internal and external, what we’re saying internally and what we’re saying to the world – that’s my job. And that’s what every person in this company is also doing. We have all these inputs, we have all these places that we could go – all these things that we could do – but we need to present one cohesive story to the world.”

A video of Jack Dorsey talking about the concept can be seen here:

Lead with product
What’s compelling to me about this is that it really orients the role of product to be about cohesive experiences first and foremost. OK, yes, there’s still schedules first, but it doesn’t drive the thing- great products drive the process.

Similarly, you don’t just jam lots of characters and plot points in a story just because. Even if they are good characters, it can bloat the story. Same with features- sometimes you have many, many good ideas for your product, but if you come to do all of them, you ultimately make it a confusing mess. Instead, you have to “edit” down the feature list until you have a clean, tight experience.

Anyway, I hope to see this trend continue in the tech industry – it sets the right tone for where we should all be focused.

Design is in.

Consumer startups no longer need to argue about product quality – it’s a prerequisite to even an initial launch. This is a good thing, but this post isn’t about that.

For social apps, what you design directly is only half the user experience. The people are just as important! So if you build a really great linksharing site that’s extremely polished and full-featured, but the community consists of Nazis, it won’t work for people.

I’m often reminded of this fact when trying XBox Live, which consists of prepubescents killing you repeated on Halo while calling you gay. The Halo content is amazing, of course, but the community around it is… um… different than me.

Dribbble as an example
Similarly, you could build a product that was an exact replica of uber-design site Dribbble, yet still fail if you didn’t have their users. Half the work is the functionality, but the other half is “designing” the right users. If you haven’t seen the rules, a lot of things have to happen before you’re allowed to actually post content there:

• Why are players drafted? http://dribbble.com/site/faq#faq-why-drafted
• “Undrafted prospects” http://dribbble.com/designers/prospects

Basically, they have a long line of “prospects” which have to be nominated by the community in order to be able to post content. They limit membership like this so that all the content on the site will only be the very best.

Eventually, opening up is key
Perhaps naturally you eventually open up and evolve beyond this, but I think at the beginning you still need a lot of authenticity.

I think the reason why this whole concept feels unfamiliar to me is that for most consumer products, the problem is getting more people, not rejecting them :) Yet at the same time, I’ve learned through a lot of first-hand experience that if you don’t curate the initial community and scale your traffic as a function of this group, you can easily fall into the trap of “designed product, but undesigned community.” That’s no good either.

Surprisingly, it can be hard to figure out if you’re at Product/Market Fit or not, and one of the big reasons is that comparable numbers are difficult or impossible to come by. You have to look at comps for products in a similar or equal product category, and sometimes they just aren’t available.

Nature versus nurture
One way to think about this is that products have a nature/nurture element to their metrics. Some product categories, like chat or email, are naturally high-frequency. You use them a lot. Other products, like tax software, might give you value but you only use it once per year. A lot of ecommerce products are in-between, where you might buy gadgets every couple months but not every day. Just because people only use your product once a year doesn’t mean you don’t have product/market fit, as long as you’re building a tax product and not chat.

The two extremes are interesting:

• Medical apps: They may have high retention since if you have a chronic ailment, you may constantly be using an app relevant to your condition, but maybe not every day
• Books/Games: You read them nonstop for a few days or a week or two, and then once you’ve consumed the content, you never go back

The point I’ll make on this is that due to the nature of certain product categories, there’s a natural range of DAU/MAUs, +1 day and +1 week retention metrics. That’s the “nature” part of the product category. No matter how good your tax software is, you won’t get people to use it every day.

Based on your product execution though, you can maximize the the metrics within the natural range. A really good news product like Flipboard is able to drive 50%+ DAU/MAUs, which are fantastic.

Some product categories cannot get high DAU/MAUs
One key conclusion of this is that it doesn’t make sense to try to compare against Twitter or Facebook’s 50% DAU/MAU unless you are in the same category as them. A lot of social games target 30% DAU/MAU, but we can also see from the Flurry chart that social games are also amongst the highest DAU/MAU categories.

That said, if you are in the same category, then these rival products really tell you how good your metrics could really be, if you executed them in the right way.

Either way, don’t fight your nature :)

Update: New chart from Flurry
A while after I wrote this, Flurry released a new version of their chart, which you can see below. Full article here. It’s interesting to see which categories have shifted a bit, I imagine because the number of new apps in each category has changed a lot.

My friend Mike Greenfield recently started blogging, and I couldn’t recommend his blog Numerate Choir more. I also had him on my list of growth hackers from a month ago. Mike is (as of today) 500 Startups’s first “Growth Hacker in Residence” and before that, co-founded Circle of Moms (acquired by Sugar), and was a data geek at Linkedin and Paypal.

Some of his excellent recent blog posts:

• Why “A/B Testing vs. Holistic UX Design” Is a False Dichotomy
• Cutting the Facebook cord
• The Visionary and the Pivoter

And with his permission, I’ve cross-posted one of his recent essays below.

Enjoy!

No, you don’t need a real-time data dashboard*
By Mike Greenfield
Originally posted on Numerate Choir

When Circle of Friends started to grow really quickly in 2007, it was really tough for Ephraim and me to stay focused.

Many times over the course of a the day, Ephraim would turn and ask me how many signups we’d had in the last ten minutes. That might have been annoying, but for the fact that I was just as curious: I’d just run the query and had an immediate answer.

Rapid viral growth can be unbelievably addictive for the people who are working to propagate it. You tweaked a key part of your flow and you want to see what kind of impact it’s having — right now. You’ve added more users in the last hour than you’ve ever added in an hour before, and you wonder if the next hour will be even better.

That addictiveness can be a great asset to growth hackers; I’d argue that anyone who doesn’t have that sort of jittery restlessness probably wouldn’t be the right fit in a growth hacking role. Restlessness is a huge motivator: I want to grow the user base, so I’m going to implement this feature and push it out as quickly as possible just so I can see what impact it will have. And if this feature doesn’t work, I’m going to try and get something else out before I leave the office so I can see if I’ve uncovered something else before it’s time to go to bed.

One day, I came up with a feature idea as I was walking to the train station in the morning. I coded it up and pushed it out while on the 35 minute train ride. There was a ten minute walk from the train station to my office; by the time I got to the office I saw that my feature was increasing invitations by around 20%.

I loved telling that story to potential engineering hires.

Here’s the thing, though: if everyone in your company behaves like that, you may acquire a huge user base, but you’ll likely never build anything of long-term value. You’ll wind up optimizing purely for short-term performance, never moving toward a strong vision

Back during that Circle of Friends growth period, I decided to automate an hourly stats email to Ephraim and myself. It satisfied our curiosity about how things were growing right now, but it stopped me from running SQL queries every five minutes. At least in theory, that meant we were focused on real work for 58 minutes every hour. In retrospect, it seems ridiculous that we needed stats updates every sixty minutes, but that actually was an improvement.

My distracted experience is why I worry about the effect of analytics companies that now promote a real-time dashboard as an awesome new feature.

It’s technically impressive that they’ve implemented real-time functionality. And at first glance, it’s very cool that I as a user can log in mid-day and see how stats are trending.

But the key distinction — and about 60% of analytics questions I’ve seen people ask over the years are on the wrong side — is if you’re looking at stats now because you’re curious and impatient, or because those stats will actually drive business decisions.

I’m afraid that in most cases, real-time stats are being used by people who aren’t iterating as quickly as growth hackers. The “need” for stats is driven more by curiosity and impatience than by decision-making.

Execs who are making big picture decisions are probably better served by looking at data less frequently. Growth hackers and IT ops types can and should attack problems restlessly — a big part of their job is optimizing everything for the immediate future. But executives are best-served waiting (perhaps until the end of the week), so they can take a long, deep look at the data and think more strategically.

This last weekend, I watched Steve Jobs: The Lost Interview (It’s available on iTunes for $3.99 rental). It’s great for many, many reasons, and I wanted to write an important point I seized upon during the talk. Here’s the link, if you want to watch it yourself.

Let’s start with an important quote:

“Insanely great”

That phrase is one of the most confusing things about the Apple philosophy, and I think it is commonly misinterpreted. Product designers often use it as an excuse to endlessly work on their product, with no release date or eye on costs. It becomes the reason why people want to focus on building completely new products and avoid copying competitors.

Apple has done a lot of stealing and reinventing
Yet in the interview, Steve Jobs has lots of interesting anecdotes:

• Apple copying the graphical user interface from Xerox PARC
• The famous quote, “Great artists steal.”
• How NeXT was building web products, same as everyone else

He says all of this, while at the same time criticizing others for lack of taste and insulting their product quality.

Great products, regardless of source
To me, the way to reconcile this is that Steve Jobs cares first and foremost about great products. Sometimes the way to get there was to steal. Sometimes you reinvent and reimagine. And sometimes, you have to invent.

The point is, building a great product is about curating from the entire space of possible features you could build. Shamelessly steal ideas when they are the best ones. Ignore bad ideas even if they’re commonplace. Don’t think you have to build something totally different to make a great product.

I think this has matched with Apple’s strategy towards their most recent generation of products – though they didn’t invent the GUI, the mouse, the MP3 player, downloadable music, the laptop, or the smartphone, they’ve build some of the best products out there. (I’ll give them a lot of great for the iPad though, which is truly a new invention)

The craving for novelty in Silicon Valley
So for all the product managers and designers out there – if you are finding yourself wanting to do it differently just because, or trying to find novel solutions just because, then maybe your priorities are not in order. The goal of building great products is for you to deliver something great to the customer, not to impress your designer friends on what new layout or interaction you’ve just developed.

Make it insanely great, even while you copy, steal, reinvent, or invent whatever you need to make that happen.

Anyway, it’s a great interview and I think everyone involved in tech products should watch it.

Originally asked on Quora. If you find yourself mostly thinking about balancing satisfaction versus virality, you’re probably doing it wrong. The Quora question is a false dilemma, because it asks you to choose between satisfaction and virality, and then quantifying the tradeoff. Most of the time, if you’re working on naturally viral products, you spend most of your time elsewhere. The world of product decisions is more like:

That is, you have features in your product that either drive growth or don’t, and you have features in your product that either really help the value proposition, or don’t. These are actually pretty independent factors and you can build product features that hit each different quadrant. For example, if you are building a product like Skype, finding your friends and sending invites is clearly a high value prop, high virality action. After all, you can’t use Skype by yourself. But if you take the exact same feature, and try to bolt it onto a non-viral product like, say, a travel search engine, then you’re just creating spam. There’s really no great reason to “find friends” in a travel product, though it might be useful to share your itinerary. A feature that’s high-value in one product is spam in the other. And if you think about each quadrant, you get something like this: Let’s talk about each bucket:

• Awesome features grow your product and also people love them. The Skype “find friends” feature is a great one, but so is Quora’s “share to Twitter” feature. After I write this post, I want people to comment and upvote, so something that lets me publish to my audience, which is both viral and part of the value prop is awesome.
• Do it anyway features are just the core of your UX. Writing on walls on Facebook may not be inherently viral in themselves, but it’s important to the product experience, keeps people coming back, and indirectly helps drive the virality of the product. The more people you have coming back, the more changes you have for them to create content or invite people
• Spam features are high virality actions that your users don’t really want to do, and don’t add to the product value prop. I think this is the bucket that the tradeoff lives of a question like, “should I be viral, or offer a great product?” If you are spending a lot of time in this quadrant, then you are shaky ground.
• WTF needs no explanation

Ideally, you want to pick a proven product category that’s naturally viral and high-retention, for instance communication, publishing, payments, photos, etc. – and then spend as much time building awesome features that both drive growth and also make your users happy. Stay away from spam features as much as you can, or use them sparingly lest your product becomes spam.

[UPDATE: I have taken a much longer and more comprehensive whack at this problem in this deck:

How to build a growth team (50 slides)

Here, I answer a couple important questions:

• Why create a growth team?
• What’s the difference between a “growth hacker” and a growth team?
• What’s the difference between growth and marketing/product/whatever?
• Where should growth teams focus?
• I’m starting or joining a growth team! What should I expect?

Hope you enjoy it!

And for the previous answer, which I typed up on Quora some time ago, you can read below.]

So what does a growth team work on day to day? I would break down what a growth team does into two major buckets:

1) Planning/modeling
2) Growth tests.

Let’s dive in, but starting with the usual caveat – you need a killer product before you should start working on growth.

But first, you need a great product
Let me note that if people aren’t using your product, then you’re wasting your time spending too much time optimizing growth. You need a base of users who are happy and then your job is to scale it.

With that caveat in mind, let’s start with the planning activities:

Planning and model building
The planning/modeling side of things is really about understanding, “Why does growth happen?” Every product is different.

• You might find that people find you via SEO and then turn into users that are retained via emails
• You might find that people come to your site via web and then cross-pollinate to mobile, and that’s the key to your growth.
• You might find you need to get them to follow a certain # of people.
• You might realize they need to clip a certain # of links to get started.

These are all things that are product-specific, so I can’t give specific advice in this answer, but this is the foundation for understanding why your product grows. You can come up with a model by looking at your flows for how users come into the site, by talking to users, and by understanding similar products. You can look at successful users and unsuccessful ones.

Once you have a good model, you can create more specific criteria in evaluating the outcome of a good or bad growth project. Your mental model doesn’t have to be perfect at first- the goal is just to get started. As you execute your project successfully, if your growth goes up, then your confidence will grow. (Or you’ll have to revisit things if you keep improving that one metric significantly but overall signups doesn’t go up)

At a more tactical level, eventually this model gets more fine-grained and you can start thinking about individual things that you can change to increase overall growth. Ideally you can model a lot of this in a spreadsheet so you can do scenario-planning around what works and what doesn’t.

The goal is to create some kind of feedback loop that results in sustained growth. Maybe you buy ads, make money, and then reinvest even more in ads. Maybe you get people to create content, driving SEO, which brings in more people that create content. Or maybe you have something invitation based. The important part is to model this process and its component parts.

Project execution
Once you have a model for how to drive your growth, the next part is to actually come up with a bunch of project ideas that can make those numbers go up and to the right. Ideally you can do lots of A/B tests for pretty short ideas that prove out the concept. If it works out, then keep investing.

For something like this, you’ll need a bit of A/B testing infrastructure, a lot of creativity, and some dedicated engineers to get the tests out there.

Because the majority of A/B tests don’t do what you want (maybe the number is <30%) as a result, you’ll want to have many, many A/B tests going at the same time so that you get a couple winners every week. Sometimes people do 1-2 A/B tests per week and then complain that it doesn’t work for them – they probably need to 5-10X their A/B test output in order to get a win or two per week.

To execute each growth project, you may also need to develop some instrumentation around tracking where users come from, and what they do. This can be a bunch of SQL databases and reporting at first, but might move to something fancier later on.

Eventually, the results of these tactical projects feed back into the uber model – you have to constantly reevaluate your priorities and understand which places in the product are the most leveraged in driving growth. So there’s a feedback loop of jumping from the strategic to the tactical, and back.

Summary
To summarize the above:

1. Have a solid product where your users are happy
2. Coming up with a model for how your site grows
3. Trying out ideas and deploying them as A/B tests
4. If the site grows, then try out more ideas. If it doesn’t, rethink the model in step 1 because it might be broken

Hope that helps.

Answered originally on Quora here.

Question: For online/mobile consumer services, in what scenarios does high organic user growth not imply product-market fit?

There’s been a bunch of recent examples of products that grow quickly but have little to no retention/engagement.

The reason is that in this context, you can think of products as having 2 main components:

• Distribution tactics: This is the viral loop – the flow within the site that generates invites, embeds, links, or otherwise exposes new users to the product. Example, for Skype, you can through to a process of inviting and build your addressbook – this generates invites.
• Product experience: The actual usage patterns of the product. For Skype, that’s chatting or talking over VOIP.

In the case of Skype, the viral loop easily flows into the product experience – as a result, you have a nice product that’s both viral and engaging. This is the good case.

Let’s talk about the dysfunctional cases though:

Viral design patterns that don’t make sense for a product
Sometimes though, you end up with a viral loop that’s pretty different/weird compared to the core product experience. For example, there’s a few design patterns that have been viral in the past:

• Filling out a quiz and comparing yourself to others
• Sending a gift or a poke to a bunch of people and then asking the recipient to poke/gift back
• Finding friends and sending invites
• Getting a notification saying that someone has a crush on you and making you fill out email addresses to guess the crush – these emails then generate the next batch of notifications
• … and newer patterns like the Social Reader design pattern on Facebook, or something like spammy low-quality SEO content, which isn’t viral but is the same kind of idea.

(Note these are less effective these days since they’ve been played out – I write about the idea of people becoming desensitized to marketing here)

Because finding a really effective, working viral loop can be rare, sometimes people build a viral loop and then bolt a product onto it. This can be done in a haphazard way that shows a lot of top-line growth but fails on retention/engagement.

Disjointed viral + product experiences
The problem that sometimes, after completing the viral actions, the experience of then using the product is too disjointed, and users bounce right away. For example, you couldn’t put a “find your friends” invitation system in front of a search engine. It doesn’t make any sense. Search engines aren’t social.

The way you could validate this was happening is just to look at the underlying stats past the top-line growth:

• After signing up, how many users are active the next day? Or the next week?
• How many users bounce after the initial viral flow?
• How aggressive is the viral loop, and do you allow the user to understand and experience the underlying product?
• How well does the viral loop communicate that it’s part of a larger, deeper product?
• Does the viral loop makes sense within the context of the product? Does completing the viral loop make the resulting product experience better?

I would look at any new product and ask the above questions to understand what’s going on. In the success case, you have a lot of retention and engagement, and the more viral the product, the stickier it gets. And ideally the design of the viral loop is very “honest” as to how it fits into the rest of the products.

Business models are important, but today they’re commoditized
Let me first state: Business models are important. Of course businesses have to make money, that’s a given. But that’s not my point – my point is:

Business models are a commodity now, so “how will they make money?” isn’t an interesting question. The answers are all obvious.

So when you see the next consumer mobile/internet product with millions of engaged users, let’s stop asking about their business model expecting a clever answer – they’ll have dozens of off-the-shelf solutions to choose from – and instead, let’s start asking about the parts of their business that aren’t commoditized yet. (More on this later)

Outsource your monetization
Between the original dotcom bubble versus now, a lot has changed for consumer internet companies. Thankfully, monetization is now a boring problem to solve because there’s a ton of different options to collect revenue that didn’t exist before:

• There’s 200+ ad networks to plug into
• Payment providers like Paypal, Amazon, Stripe
• “Offer walls” like Trialpay
• Mobile payment solutions like Boku
• … and new services coming out all the time (Kickstarter)

Not only that, consumers know and expect to pay for services, something that was novel back in the late 90s. If you offer some sort of marketplace like Airbnb, they’ll expect a listing fee. If you are making a social game on Facebook, they’ll expect to be able to buy more virtual stuff. They’ll expect to pay $0.99 for an iPhone app.

Contrast this with the dotcom bubble, in which you were creating brand new user behavior as well as building these monetization services in-house. In eBay’s case, people just mailed each other (and eBay) money for their listings. Small websites had to build up ad sales teams in order to get advertising revenue, instead of plugging into ad networks. Building apps for phones involved months of negotiation with carriers to get “on deck.” At my last startup, an ad targeting technology company, we encountered companies like ESPN which had written their own ad servers because they didn’t have off-the-shelf solutions when they first started their website back in the late 1990s.

Let me repeat that: They wrote their own ad server as part of building their news site. And that means they had engineers writing lots of code to support their business model rather than making their product better.

Product experience renaissance
Let’s be thankful that we don’t all have to build an ad server every time our Ruby on Rails app is successful. This lets consumer product companies focus on what they’re best at. Also, building a new website doesn’t require $5M anymore. The number of risks in getting your company off the ground are vastly reduced when you combine cheap server hosting, an open source software stack, and multiple bolt-on revenue streams.

This frees us up to be able to work on what’s really important: Building and marketing great products.

These days, the primary cost for any pre-traction company is the apartment rent of the developers who are coding up the product. The profitability of any post-traction company is just based on how fast the team wants to ramp up headcount. If a team can hit product/market fit, a lot of other problems are taken care of.

The lesson behind Facebook’s $3.7B in revenue
Once upon a time, I was skeptical about Facebook’s business model because they received a mere 0.2 cents in advertising revenue per pageview they generated. In 2006, I calculated that maybe they could generate $15M in revenue per year maximum – a nice business, but not a world-changing one. I wrote about this topic here: Why I doubted Facebook could build a billion dollar business, and what I learned from being horribly wrong.

As I wrote in my post, it turns out I was wrong, and Facebook in fact generated $3.7B in 2011 and will generate more than $5B this year. I was wrong in an interesting way though – it turns out that they didn’t dramatically increase their revenue per pageview, but rather they just grew and grew and grew, to ~1 trillion pageviews/month. My mental model was all wrong.

In fact, we have a lot more experience with advertising and transaction based models. It’s pretty clear that an engaging social website will have 0.1% to 0.5% CTRs on their ads, and net an average $0.50 CPM. If you sell something, or have a freemium site, then you can expect 0.5% to 1% of your active users to convert. There’s lots of benchmarks out there, which I discuss in this older blog post. The point is, if you have the audience, you can find the revenue – it’s getting the big audience that’s the main problem.

The last dotcom bubble conditioned many of us to think about a different world than the one we face today. In 1997, there were a mere ~100M users on the internet, mostly on dialup modems. Let me repeat that: The entire dotcom bubble, with all of its bubbly goodness, was based off of 100M dialup users. Compare that to today, where we have 20X that number, over 2 billion users on broadband and mobile. The graph, courtesy World Bank via Google, is incredible.

The point is, the consumer market has grown by so much that the upside opportunity is tremendous if you get a product exactly right. Given all the growth opportunity, and given the plug-in revenue models, the main bottleneck for building a great company doesn’t seem to be the business model at all.

In fact, the business model seems like a second or third order problem. So again, I argue, let’s stop asking about it.

At over 450 million uniques per month, let’s stop wondering what Twitter’s revenue model will be. Obviously it will be some form of advertising, and maybe they’ll experiment with freemium or transaction fees somehow. You can debate if you think they will ultimately be a $100B company or a $10B one, but let’s skip the conversation on whether or not they’ll fail because they don’t have a business model.

The new question to ask
If you agree with me that business model is no longer a first-order question, then what’s the real question to ask? The thing that makes the business model work is really about getting to the scale where the business model becomes trivial.

Let’s ask a more important question:

Could this product engage and retain 100s of millions of active users?

For the first time ever, hitting 100+ million active users is actually realistic. First off, how incredible is that? In recent years, many startups have done it, such as: Zynga, Facebook, Twitter, Groupon, Linkedin, etc. I think we’ll also see Dropbox, Pandora, and others get there too.

For an early stage company, asking this question is really just a test of the team’s ambition, their initial market, and an evaluation of their product/market fit. Obviously if their product isn’t working, they won’t even be close.

Once a startup has product/market fit and is scaling, then the answer to this question revolves around marketing and technology competence. Also, the product might have to evolve as the initial market gets saturated- like Facebook with college and Twitter with their early adopter audience.

To sum this all up:

• Making money as a business is important, but commoditized
• You can plug into 100s of options for monetizing an audience, if you have one
• We’re working with 20X the internet audience compared to the dotcom bubble, and 1/10 the cost of starting a company
• Facebook is hitting $5B in revenue via sheer growth, not monetization innovation
• You should aim to hit 100 million active users, and get an off-the-shelf monetization solution later
• Evaluate new companies on market size and ability to grow to 100 million actives, rather than monetization methods

Metrics are merely a reflection of the product strategy that you have in place
Data is powerful because it is concrete. For many entrepreneurs, particularly with technical backgrounds, empirical data can trump everything else – best practices, guys with fancy educations and job titles – and for good reason. It’s really the skeptic’s best weapon, and it’s been an important tool in helping startups solve problems in new and innovative ways.

It’s easy to go too far – and that’s the distinction made between “data-informed” versus “data-driven,” which I originally heard at a Facebook talk in 2010 (included underneath the post). Ultimately, metrics are merely a reflection of the product strategy that you already have in place and are limited because they’re based on what you’ve already built, which is based on your current audience and how your current product behaves. Being data-informed means that you acknowledge the fact that you only have a small subset of the information that you need to build a successful product. After all, your product could target other audiences, or have a completely different set of features. Data is generated based on a snapshot based on what you’ve already built, and generally you can change a few variables at a time, but it’s limited.

This means you often know how to iterate towards the local maximum, but you don’t have enough data to understand how to get to the best outcome in the biggest market.

This is a messy problem, don’t let data falsely simplify it
So the difference between data-informed versus data-driven, in my mind, is that you weigh the data as one piece of a messy problem you’re solving with thousands of constantly changing variables. While data is concrete, it is often systematically biased. It’s also not the right tool, because not everything is an optimization problem. And delegating your decision-making to only what you can measure right now often de-prioritizes more important macro aspects of the problem.

Let’s examine a couple ways in which a data-driven approach can lead to weak decision-making.

Data is often systematically biased in ways that are too expensive to fix
The first problem with being data-driven is that the data you can collect is often systematically biased in unfixable ways.

It’s easy to collect data when the following conditions are met:

• You have a lot of traffic/users to collect the data
• You can collect the data quickly
• There are clear metrics for what’s good versus bad
• You can collect data with the product you have (not the one you wish you had)
• It doesn’t cost anything

This type of data is good for stuff like, say, signup %s on homepages. They are often the most trafficked parts of the site, and there’s a clear metric, so you can run an experiment in a few days and get your data back quickly.

In contrast, if you are looking to measure long-retention rates, that’s much more difficult. Or long-term perceptions of your user experience, or trying to measure the impact of an important but niche feature (like account deletion). These are all super difficult because they take a long time, or are expensive, or are impossible datapoints to collect – people don’t want to wait around for a month to see what their +1 month retention looks like.

And yet, oftentimes these metrics are exactly the most important ones to solve.

Worse yet, consider the cases where you take a “data-driven” mindset and try to trade off the metrics between concrete datapoints like signup %s versus long-term retention rates. It’s difficult for retention to ever win out, unless you take a more macro and enlightened perspective on the role of data. Short- vs long-term tradeoffs require deep thinking, not shallow data!

Not everything is an optimization problem
At a more macro level, it’s also important to note that the most important strategic issues are not optimization problems. Let’s start at the beginning, when you’re picking out your product. You could, for example, build a great business targeting consumers or enterprises or SMBs. Similarly, you can build businesses that are web-first (Pinterest!) or mobile-first (Instagram!) and both be successful. These are things where it might be nice to have a feel for some of the general parameters, like market size or mobile growth, but ultimately they are such large markets that it’s important to make the decision where you feel good about it. In these cases, you’re forced to be data-informed but it’s hard to be data-driven.

These types are strategy questions are especially important when the industry is undergoing a disruptive innovation, as discussed in Innovator’s Dilemma. In the book, Clayton Christensen discusses the pattern of companies who are successful and build a big revenue base in one area. They find that it’s almost always easier to increase their core business by 10% than it is to create a new business to do the same, but this thinking eventually leads to their demise. This happened in the tech industry from mainframes vs PCs, hardware vs software, desktop vs web, and web vs mobile now. The incumbents are doing what they think is right- listening to their current customer base, improving revenues from a % basis, and in general trying to do the most data-driven thing. But without a vision for how the industry will evolve and improve, the big guys are eventually disrupted.

Leverage data in the right way
It’s important to leverage data the same way, whether it’s a strategic or tactical issue: Have a vision for what you are trying to do. Use data to validate and help you navigate that vision, and map it down into small enough pieces where you can begin to execute in a data-informed way. Don’t let shallow analysis of data that happens to be cheap/easy/fast to collect nudge you off-course in your entrepreneurial pursuits.

Facebook on data-informed versus data-driven
I leave you with the Facebook video that inspired this post in the first place – presented by Adam Mosseri. He uses the example of multiple photo uploads, and how they use metrics to optimize the workflow. Watch the video embed below or go to YouTube.

Don Valentine, who founded Sequoia Capital, talks about what makes Sequoia Capital effective. It’s one of my favorite talks, and I find myself watching and re-watching it from time to time, and I’d encourage everyone to hear the wisdom themselves.

Markets, not team
In the beginning of the video, Don Valentine asks, why is Sequoia successful? He says that most VCs talk about how they finance the best and the brightest, but Sequoia focuses instead on the size of the market, the dynamics of the market, and the nature of the competition.

This is, of course, super interesting because in many ways it’s contrarian to the typical response that investing is all about “team.”

Creating markets versus exploiting markets
Another choice quote: “We’re never interested in creating markets – it’s too expensive. We’re interested in exploiting markets early.”

In consumer internet, when the divisions that separate product categories are so fuzzy, it can be hard to understand when you’re creating a market versus when you’re attacking an existing one. My rule of thumb is that:

If people know how to search for products in your category then you are in an existing market.

I’ve written more about this in posts here and here

Watch the video of Don Valentine of Sequoia capital on “Target Big Markets” on YouTube or in the embed below:

Using retweets to assess content virality
Recently I’ve been running an experiment:

1. Tweet an insight, idea, or quote
2. See how many people retweet it
3. If it catches, then write a blog post elaborating on the topic

My recent Growth Hacker post was the result of one such tweet, which you can see above in my Crowdbooster dashboard. I wrote it on a whim, but after the retweets, I developed it into a longer and more comprehensive blog post. (Note that sometimes a tweet is not suitable to developed into a blog post, but most of the time this technique works)

Why this works
This works because the headline is key. It spreads the content behind it.

This is especially true on Twitter, but it’s also true for news sites that will pick up and syndicate your content. If that headline is viral and the content behind it is high quality, there’s a multiplier effect – sometimes a difference of 100X or more. Naturally, you want to optimize the flow of how people interact with your content, starting with what they see first: The title.

After all, what’s a better test for whether the following will be viral:

New blog post: Growth Hacker is the new VP Marketing [link]

than the tweet:

Growth Hacker is the new VP marketing

It’s a natural test.

I’ll also argue that if you can express the core of your idea in a short, pithy tweet, then that’s a good test for whether the underlying blog post will be interesting as well. Great tweets are often provocative insights or mesmerizing quotes, and there’s a lot to say by examining the issues more deeply. Contrast this to writing a long, unfocused, laundry-list essay examining a topic from all angles, taking no interesting positions or risks along the way – now that’s a recipe for boredom.

Combining virality with a high-quality product, of course, is the key to a lot of things – not just blogging :)

Don’t waste your time writing what people don’t want to read
Testing your ideas like this allows you to invest more time and effort into the content – a clear win.

Personally, I love writing long-form content that dives deep into an area, and also enjoy reading it as well. Unfortunately, writing a blog post often takes a long time – an hour or more. Use this technique to make it safer to spend more time, think more deeply, and research more broadly on you write. In my experience, writing a high-quality, highly retweetable blog post once per month is better than writing a daily stream of short, low-quality posts that no one will read. Plus, it takes less time.

As a smart guy once said: “Do less, but better.”

I recently asked, and then answered my own question on Quora and wanted to share here as well.

Has Facebook’s DAU/MAU always been ~50%?

According to public info, Facebook’s DAU/MAU is 58% these days. Here’s a link.

It states:

• 901 million monthly active users at the end of March 2012
• 526 million daily active users on average in March 2012

Has Facebook’s DAU/MAU always been this good, as a consequence of its product category (communication/photo-sharing/etc.)? Or was it once a lot worse and was improved over time?

(UPDATE: Here’s a followup question I have about the same topic- Was Facebook’s DAU/MAU ~50% prior to launching the Newsfeed in 2009?)

Answer: Yes, Facebook’s DAU/MAU has been close to 50%, at least since 2004.

Based on their media kit from 2004, their DAU/MAU was already 75%.

Since this media kit, their DAU/MAU data has been included in their financials since 2009. However, I theorize that Facebook’s DAU/MAU has always been high as a natural outcome of the communication-oriented usage of the product. Contrast this to a product category like ecommerce which you are unlikely to use and purchase with every day.

In their recent financial filings, the following chart is shown for Facebook’s DAU and MAU since 2009:

If you do a graph of the DAU/MAU on this data, since 2009, you’ll see that it starts around 45-47% and goes up to a very impressive 58% recently.

(As an aside, another interesting aspect is that Facebook’s MAU growth looks pretty much like a straight line, and so the % growth has been slowing down as of late. The MAU growth was around 23% starting in 2009, but is now down to 6-7% in recent months. See below for a graph on MAU vs % MAU growth)

After writing my recent article on Growth Hackers, I’ve been asked by quite a few folks on how to learn the discipline. The best answer is, learn from someone who’s already good at it – if you’re technical and creative, it’s well worth the time.

I would encourage everyone to also read Andy Johns’s Quora answers on What is Facebook’s User Growth team responsible for and what have they launched? and
What are some decisions taken by the “Growth team” at Facebook that helped Facebook reach 500 million users?– it lays out a lot of the key activities used in a well-run growth team.

The list below includes some of these folks I know personally, some just by reputation- but collectively they’ve grown products up to millions, 10s of millions, and in some cases, 100M+ users. Typically they use quantitatively-oriented techniques centered on virality across different channels such as iOS, Facebook, email, etc. There’s lots of iteration, A/B testing, and experimentation involved. There’s also really great growth hackers centered around SEO, SEM/ad arb, and other techniques, but for the most part I’m just listing out the folks around quant-based virality. The important thing about virality is, it’s free :) So it’s an important skill for startups.

Missing from this list are many unsung heroes over at Zynga, Dropbox, Branchout, Viddy/Socialcam, lots of ex-Paypal/Slide people, etc., etc. Also, all of these guys typically have co-founders or entire growth teams around them that are experts, even if I don’t know them by name.

If others in the community would like to make suggestions, tweet me at @andrewchen or just reply in the comments.

UPDATE: My friend Dan Martell’s new company, Clarity, provides a way to access experts like this via phone and email. Here’s the directory of folks with expertise on growth.

The rise of the Growth Hacker
The new job title of “Growth Hacker” is integrating itself into Silicon Valley’s culture, emphasizing that coding and technical chops are now an essential part of being a great marketer. Growth hackers are a hybrid of marketer and coder, one who looks at the traditional question of “How do I get customers for my product?” and answers with A/B tests, landing pages, viral factor, email deliverability, and Open Graph. On top of this, they layer the discipline of direct marketing, with its emphasis on quantitative measurement, scenario modeling via spreadsheets, and a lot of database queries. If a startup is pre-product/market fit, growth hackers can make sure virality is embedded at the core of a product. After product/market fit, they can help run up the score on what’s already working.

This isn’t just a single role – the entire marketing team is being disrupted. Rather than a VP of Marketing with a bunch of non-technical marketers reporting to them, instead growth hackers are engineers leading teams of engineers. The process of integrating and optimizing your product to a big platform requires a blurring of lines between marketing, product, and engineering, so that they work together to make the product market itself. Projects like email deliverability, page-load times, and Facebook sign-in are no longer technical or design decisions – instead they are offensive weapons to win in the market.

The stakes are huge because of “superplatforms” giving access to 100M+ consumers
These skills are invaluable and can change the trajectory of a new product. For the first time ever, it’s possible for new products to go from zero to 10s of millions users in just a few years. Great examples include Pinterest, Zynga, Groupon, Instagram, Dropbox. New products with incredible traction emerge every week. These products, with millions of users, are built on top of new, open platforms that in turn have hundreds of millions of users – Facebook and Apple in particular. Whereas the web in 1995 consisted of a mere 16 million users on dialup, today over 2 billion people access the internet. On top of these unprecedented numbers, consumers use super-viral communication platforms that rapidly speed up the proliferation of new products – not only is the market bigger, but it moves faster too.

Before this era, the discipline of marketing relied on the only communication channels that could reach 10s of millions of people – newspaper, TV, conferences, and channels like retail stores. To talk to these communication channels, you used people – advertising agencies, PR, keynote speeches, and business development. Today, the traditional communication channels are fragmented and passe. The fastest way to spread your product is by distributing it on a platform using APIs, not MBAs. Business development is now API-centric, not people-centric.

Whereas PR and press used to be the drivers of customer acquisition, instead it’s now a lagging indicator that your Facebook integration is working. The role of the VP of Marketing, long thought to be a non-technical role, is rapidly fading and in its place, a new breed of marketer/coder hybrids have emerged.

Airbnb, a case study
Let’s use case of Airbnb to illustrate this mindset. First, recall The Law of Shitty Clickthroughs:

Over time, all marketing strategies result in shitty clickthrough rates.

The converse of this law is that if you are first-to-market, or just as well, first-to-marketing-channel, you can get strong clickthrough and conversion rates because of novelty and lack of competition. This presents a compelling opportunity for a growth team that knows what they are doing – they can do a reasonably difficult integration into a big platform and expect to achieve an advantage early on.

Airbnb does just this, with a remarkable Craigslist integration. They’ve picked a platform with 10s of millions of users where relatively few automated tools exist, and have created a great experience to share your Airbnb listing. It’s integrated simply and deeply into the product, and is one of the most impressive ad-hoc integrations I’ve seen in years. Certainly a traditional marketer would not have come up with this, or known it was even possible – instead it’d take a marketing-minded engineer to dissect the product and build an integration this smooth.

Here’s how it works at a UI level, and then we’ll dissect the technology bits:

(This screenshots are courtesy of Luke Bornheimer and his wonderful answer on Quora)

Looks simple, right? The impressive part is that this is done with no public Craigslist API! It turns out, you have to look closely and carefully at Craigslist in order to accomplish an integration like this. Note that it’s 100X easier for me to reverse engineer something that’s already working versus coming up with the reference implementation – and for this reason, I’m super impressed with this integration.

Reverse-engineering “Post to Craigslist”
The first thing you have to do is to look at how Craigslist allows users to post to the site. Without an API, you have to write a script that can scrape Craigslist and interact with its forms, to pre-fill all the information you want.

The first thing you can notice from playing around with Craigslist is that when you go to post something, you get a unique URL where all your information is saved. So if you go to https://post.craigslist.org you’ll get redirected to a different URL that looks like https://post.craigslist.org/k/HLjRsQyQ4RGu6gFwMi3iXg/StmM3?s=type. It turns out that this URL is unique, and all information that goes into this listing is associated to this URL and not to your Craigslist cookie. This is different than the way that most sites do it, where a bunch of information is saved in a cookie and/or server-side and then pulled out. This unique way of associating your Craigslist data and the URL means that you can build a bot that visits Craigslist, gets a unique URL, fills in the listing info, and then passes the URL to the user to take the final step of publishing. That becomes the foundation for the integration.

At the same time, the bot needs to know information to deal with all the forms – beyond filling out the Craigslist category, which is simple, you also need to know which geographical region to select. For that, you’d have to visit every Craigslist in every market they serve, and scrape the names and codes for every region. Luckily, you can start with the links in the Craiglist sidepanel – there’s 100s of different versions of Craigslist, it turns out.

If you dig around a little bit you find that certain geographical markets are more detailed than others. In some, like the SF Bay Area, there’s subareas (south bay, peninsula, etc.) and neighborhoods (bernal, pacific heights) whereas in other markets there’s only subareas, or there’s just the market. So you’d have to incorporate all of that into your interface.

Then there’s the problem of the listing itself – by default, Craigslist works by giving you an anonymous email address which you use to communicate to potential customers. If you want to drive them to your site, you’d have to notice that you can turn off showing an email, and just provide the “Contact me here” link instead. Or, you could potentially fill a special email address like listing-29372@domain.com that automatically directs inquiries to the right person, which can be done using services like Mailgun or Sendgrid.

Finally, you’ll want the listing to look good – it turns out Craigslist only supports a limited amount of HTML, so you’ll need to work to make your listings work well within those constraints.

Completing the integration is only the beginning – once it’s up, you’d have to optimize it. What’s the completion % once sometime starts sharing their listing out to Craigslist? How can you change the flow, the call to action, the steps in the form, to increase this %? And similarly, when people land from Craigslist, how do you make sure they are likely to complete a transaction? Do they need special messaging?

Tracking all of this requires additional work with click-tracking with unique URLs, 1×1 GIFs on the Craigslist listing, and many more details.

Long story short, this kind of integration is not trivial. There’s many little details to notice, and I wouldn’t be surprised if the initial integration took some very smart people a lot of time to perfect.

No traditional marketer would have figured this out
Let’s be honest, a traditional marketer would not even be close to imagining the integration above – there’s too many technical details needed for it to happen. As a result, it could only have come out of the mind of an engineer tasked with the problem of acquiring more users from Craigslist. Who knows how much value Airbnb is getting from this integration, but in my book, it’s damn impressive. It taps into a low-competition, huge-volume marketing channel, and builds a marketing function deeply into the product. Best of all, it’s a win-win for everyone involved – both the people renting out their places by tapping into pre-built demand, and for renters, who see much nicer listings with better photos and descriptions.

This is just a case study, but with this type of integration, a new product is able to compete not just on features, but on distribution strategy as well. In this way, two identical products can have 100X different outcomes, just based on how well they integrate into Craigslist/Twitter/Facebook. It’s an amazing time, and a new breed of creative, technical marketers are emerging. Watch this trend.

So to summarize:

• For the first time ever, superplatforms like Facebook and Apple uniquely provide access to 10s of millions of customers
• The discipline of marketing is shifting from people-centric to API-centric activities
• Growth hackers embody the hybrid between marketer and coder needed to thrive in the age of platforms
• Airbnb has an amazing Craigslist integration

Good luck, growth hackers!

I was reading today’s NYT article on Google+’s new redesign and found myself continually puzzled by the key metric Google continues to report as the success of their new social product: Registered Users.

In the very first sentence, Vic Gundotra writes:

More than 170 million people have upgraded to Google+, enjoying new ways to share in Search, Gmail, YouTube and lots of other places.

The use of registered users is a vanity metric, and reflects how easily Google can cross-sell any new product to their core base of 1 billion uniques per month. What it doesn’t reflect, however, is the actual health of the product.

Ultimately, this misalignment of metrics is due to the curse of instant distribution. Because Google can cross-sell whatever products they want against their billion unique users, it’s easy to grade on that effort. Plus it’s a big number, who doesn’t love a big number?

Google+ should be measured on per user metrics
Here’s what metrics are more important instead: Given the Google+ emphasis on Circles and Hangouts, you’d think that the best metrics to use would evaluate the extent to which these more personal and more authentic features are being used. These would include metrics like:

• Shares per user per day (especially utilizing the Circles feature)
• Friends manually added to circles per user per day (not automatically!)
• Minutes of engagement per user per day

Point is, the density and frequency of relationships within small circles ought to matter more than the aggregate counts on the network. As I’ve blogged about before, you use metrics to reflect the strategy you already have in place, and based on the Google+’s focus on authentic circles of friends, you’d think the metrics would focus on the density of friendships and activities, and not the aggregate numbers.

The curse of instant distribution
Every new product for a startup goes through a gauntlet to reach product/market fit, and then traction. In the real world, product quality and the ability to solve a real problem for people ends up correlating with your ability to distribute the product. Google+ is blessed, and cursed, with the ability to sidestep this completely. They are able to onboard hundreds of millions of users without having great product/market fit, and can claim positive metrics without going through the gauntlet of really making their product work.

Adam D’Angelo of Quora (and previously CTO of Facebook) wrote this insightful commentary regarding Google Buzz a while back:

Why have social networks tied to webmail clients failed to gain traction?
Personally I think this is mostly because the social networking products built by webmail teams haven’t been very good. Even Google Buzz, which is way ahead of the attempts built into Yahoo Mail and Hotmail, has serious problems: the connections inside it aren’t meaningful, profiles and photos are second class, comments bump items to the top of the feed meaning there’s old stuff endlessly getting recycled, and the whole product itself is a secondary feature accessible only through a click below the inbox, which hasn’t gotten it enough distribution to kick off and sustain conversations.

I’m pretty sure that if Google, Microsoft, or Yahoo had cloned Facebook almost exactly (friends, profiles, news feed, photos) and integrated it well into their webmail product, that it could have taken off (before Facebook got to its current scale; at this point it will be hard for any competitor, even with a massive distribution channel pushing it).

So I think this question is really, why are social networks that webmail teams build always bad? Here’s my guess:

• The team building the social network knows that they’re going to get a huge amount of distribution via the integration and so they aren’t focused on growth and making a product that people will visit on their own.
• Integrating any two big products is really hard.
• Any big webmail provider is going to have a big organization behind it, and lots of politics and compromises probably make it difficult to execute well.
• Teams that work on webmail products have gotten good at building a webmail product, and haven’t selected for the skills and culture that a team that grows around building a social network will have.

(The bolding is from me). I couldn’t agree more with this answer. I think a key lesson behind the recent success of products like Instagram and Pinterest is that there’s still a lot of room in the market for great social products to take off- but the emphasis has to be on the product rather than the superficial act of onboarding a lot of new users into Google+.

Ultimately, it comes down to how realistic the Google+ folks are in looking at their metrics. If they drink their own kool-aid and think they have product/market fit when it’s in fact the traction is solely dependent on the power of their distribution channels, they may never get their product working.

On the other hand, if they have a balanced view on their metrics and know they don’t have product/market fit yet, then they have a fighting chance. Unfortunately, I think the changes they’ve made to the product recently are more efforts to optimize, rather than fundamental improvements to the product. I think Google+ needs much bigger changes to make it as engaging as the best social products.

The first banner ad ever, on HotWired in 1994, debuted with a clickthrough rate of 78% (thanks @ottotimmons)

First it works, and then it doesn’t
After months of iterating on different marketing strategies, you finally find something that works. However, the moment you start to scale it, the effectiveness of your marketing grinds to a halt. Sound familiar?

Welcome to the Law of Shitty Clickthroughs:

Over time, all marketing strategies result in shitty clickthrough rates.

Here’s a real example – let’s compare the average clickthrough rates of banner ads when debuted on HotWired in 1994 versus Facebook in 2011:

• HotWired CTR, 1994: 78%
• Facebook CTR, 2011: 0.05%

That’s a 1500X difference. While there are many factors that influence this difference, the basic premise is sound – the clickthrough rates of banner ads, email invites, and many other marketing channels on the web have decayed every year since they were invented.

Here’s another channel, which is email open rates over time, according to eMarketer:

While this graph shows a decline, the other graph (which I don’t have handy) is that the number of emails sent out has increased up to 30+ billion per day.

All these channels are decaying over time, and what’s saving us is the new marketing channels are constantly getting unveiled, too. These new channels offer high performance, because of a lack of competition, big opportunities for novel marketing techniques, and these days, the cutting edge is about optimizing your mobile notifications, not your banner placements.

There are a few drivers for the Law of Shitty Clickthroughs, and here’s a summary of the top ones:

• Customers respond to novelty, which inevitably fades
• First-to-market never lasts
• More scale means less qualified customers

Novelty
Without a doubt, one of the key drivers of engagement for marketing is that customers respond to novelty. When HotWired showed banner ads for the first time in history, people clicked just to check out the experience. Same for being the first web product to email people invites to a website – it works for a while, until your customers get used to the effect, and start ignoring it.

One of the most important tools you have at your disposal is the creative and calls to action that you use in your marketing – this might be like “X has invited you to Y” or it might be the headline you use in your banner ads. Recently, Retargeter posted an interesting analysis on the Importance of Rotating Creatives, which showed how keeping the same ad creative led to declining CTRs over time:

Publishers often have a similar problem in consumers ignoring the advertising on their site, which drives down clickthrough rates for both of them (bad for CPMs). This problem is often described as banner blindness, and you can see it clearly here in an eye-tracking study by Jakob Nielsen:

You can see here how users, almost comically, avoid looking at any banners.

The point is, humans seek novelty yet are pattern-recognition machines. Your initial marketing strategy will work quite well as your users try it for the first time, but afterwards, they learn to filter your marketing efforts out unless they are genuinely useful (more on that later).

First-to-market never lasts
It’s bad enough that your own marketing efforts drive down channel performance, but usually once your marketing efforts are working, your competitors quickly follow. There’s a whole cottage industry of companies that provide competitive research in the area of how their competitors are advertising and give you the information needed to fast-follow their marketing efforts.

For example, with a quick query, I know how much Airbnb is spending on search marketing (turns out, millions per year) what keywords they are buying ads on, and who their competitors are. And this is just a free service! There are much more sophisticated products for every established marketing channel:

Airbnb Search Engine Marketing

• Daily ad budget: $10,638
• Keywords: 62,729
• Example ad: Find Affordable Rooms Starting From $20/Day. Browse & Book Online Now!
• Main competitors: Expedia.com, booking.com, hotels.com, Marriott.com

Any clone of their business can quickly fast-follow their marketing efforts and use the same ads in the same marketing channels. This quickly degrades the performance of the marketing channel as the novelty wears off and clickthroughs plummet.

Any product that is first to market has a limited window where they will enjoy unnaturally high marketing performance, until the competition enters, in which case everyone’s marketing efforts will degrade.

More scale means less qualified customers
Another important way to think about the available market for your product is in terms of the popular Technology Adoption Lifecycle, in which early adopters actively seek out your product, while the rest of the mainstream market needs a lot of convincing. The quant marketing way to look at this is that early adopters respond better to marketing efforts across any given metric (signup %, CTR, CPA) than the later customer segments. In the TAL framework, the early market seeks out novelty, whereas the mainstream market just cares if you solve a problem for them.

As a result, a marketing strategy focused on early adopters is bound to look better than what you get later. You can get some limited traffic from PR and targeted advertising from niche communities and media properties. However once you get past this group, the CTRs can drop substantially.

If you’re a SaaS or ecommerce company that’s road-tested your marketing strategy by acquiring limited batches of customers, the problem is that whatever assumptions and projections you make off of this base end up fundamentally skewed positive. If your model indicates that you can acquire customers at $10 and break even within 6 months, it’s not hard for a 30% increase in CAC and 30% decrease in LTV to double the time it takes to get to profitability. This could be the difference between life and death for a company.

Lesson to investors is: Beware marketing metrics done at a small scale, and beware marketing tech companies that facilitate momentary marketing opportunities without a bigger vision. These are arbitrage opportunities that will disappear over time.

How to fight the Law of Shitty Clickthroughs
I call it a Law, of course, because I really believe it’s a strong gravitational pull on all marketing on the web. You can’t avoid it, and in many ways, it’s counter productive to try.

You can always get incrementally better performance out of your marketing by taking a nomad strategy – always keep developing new creative, testing new publishers, and so on. That’s all easy, but is mostly about maintaining some base level of performance. This can push the Law of Shitty Clickthroughs to act over years rather than degrading your marketing efforts over months.

Similarly, this law provides a litmus test as to the difference between advertising and information. When you are marketing with useful information, then CTRs stay high. Advertising that’s just novelty and noise wrapped in a new marketing channel has a limited shelf life.

The real solution: Discover the next untapped marketing channel
The 10X solution to solving the Law of Shitty Clickthroughs, even momentarily, is to discover the next untapped marketing channel. In addition to doubling down on traditional forms of online advertising like banners, search, and email, it’s important to work hard to get to the next marketing channel while it’s uncontested.

Sometimes I get asked “have you ever seen someone do XYZ to acquire customers?” Turns out, the highest vote of confidence I can give is, “No I haven’t, and that’s good – that means there’s a higher chance of it working. You should try it.”

Today, these (relatively) uncontested marketing channels are Open Graph, mobile notifications, etc. If you can make these channels work with a strong product behind it, then great. Chances are, you’ll enjoy a few months if not a few years of strong marketing performance before they too, slowly succumb.

I had a nerdy conversation on what might be the next mainstream framework for building web products, and in particular whether the node.js community would ultimately create this framework, or if node.js will just be a fad. This blog post is a bit of a deviation from my usual focus around marketing, so just ignore if you have no interest in the area.

Here’s the summary:

• Programming languages/frameworks are like marketplaces – they have network effects
• Rails, PHP, and Visual Basic were all successful because they made it easy to build form-based applications
• Form-based apps are a popular/dominant design pattern
• The web is moving to products with real-time updates, but building real-time apps hard
• Node.js could become a popular framework by making it dead simple to create modern, real-time form-based apps
• Node.js will be niche if it continues to emphasize Javascript purity or high-scalability

The longer argument below:

Large communities of novice/intermediate programmers are important
One of the biggest technology decisions for building a new product is the choice of development language and framework. Right now for web products, the most popular choice is Ruby on Rails – it’s used to build some of the most popular websites in the world, including Github, Scribd, Groupon, and Basecamp.

Programming languages are like marketplaces – you need a large functional community of people both demanding and contributing code, documentation, libraries, consulting dollars, and more. It’s critical that these marketplaces have scale – it needs to appeal to the large ecosystem of novices, freelancers and consultants that constitute the vast majority of programmers in the world. It turns out, just because a small # of Stanford-trained Silicon Valley expert engineers use something doesn’t guarantee success.

Before Rails, the most popular language for the web was PHP, which had a similar value proposition – it was easy to build websites really fast, and it was used by a large group of novice/intermediate programmers as well. This includes a 19-yo Mark Zuckerberg to build the initial version of Facebook. Although PHP gained the reputation of churning out spaghetti code, the ability for people to start by writing HTML and then start adding application logic all in one file made it extremely convenient for development.

And even before Rails and PHP, it was Visual Basic that engaged this same development community. It appealed to novice programmers who could quickly set up an application by dragging-and-dropping controls, write application logic with BASIC, etc.

I think there’s a unifying pattern that explains much of the success of these three frameworks.

The power of form-based applications
The biggest “killer app” for all of these languages is how easy it is to build the most common application that mainstream novice-to-intermediate programmers are paid to build: Basic form-based applications.

These kinds of apps let you do a some basic variation of:

• Give the user a form for data-entry
• Store this content in a database
• Edit, view, and delete entries from this database

It turns out that this describes a very high % of useful applications, particularly in business contexts including addressbooks, medical records, event-management, but also consumer applications like blogs, photo-sharing, Q&A, etc. Because of the importance of products in this format, it’s no surprise one of Visual Basic’s strongest value props was a visual form building tool.

Similarly, what drove a lot of the buzz behind Rails’s initial was a screencast below:

How to build a blog engine in 15 min with Rails (presented in 2005)

Even if you haven’t done any programming, it’s worthwhile to watch the above video to get a sense for how magical it is to get a basic form-based application up and running in Rails. You can get the basics up super quickly. The biggest advantages in using Rails are the built-in data validation and how easy it is to create usable forms that create/update/delete entries in a database.

Different languages/frameworks have different advantages – but easy form-based apps are key
The point is, every new language/framework that gets buzz has some kind of advantage over others- but sometimes these advantages are esoteric and sometimes they tap into a huge market of developers who are all trying to solve the same problem. In my opinion, if a new language primarily helps solve scalability problems, but is inferior in most other respects, then it will fail to attract a mainstream audience. This is because most products don’t have to deal with scalability issues, though there’s no end to programmers who pick technologies dedicated to scale just in case! But much more often than not, it’s all just aspirational.

Contrast this to a language lets you develop on iOS and reach its huge audience – no matter how horrible it is, people will flock to it.

Thus, my big prediction is:

The next dominant web framework will be the one that allows you to build form-based apps that are better and easier than Rails

Let’s compare this idea with one of the most recent frameworks/languages that has gotten a ton of buzz is node.js. I’ve been reading a bit about it but haven’t used it much – so let me caveat everything in the second half with my post with that. Anyway, based on what I’ve seen there’s a bunch of different value props ascribed to its use:

• Build server-side applications with Javascript, so you don’t need two languages in the backend and frontend
• High-performance/scalability
• Allows for easier event-driven applications

A lot of the demo applications that are built seem to revolve around chat, which is easy to build in node but harder to build in Rails. Ultimately though, in its current form, there’s a lot missing from what would be required for node.js to hit the same level of popularity as Rails, PHP, or Visual Basic for that. I’d argue that the first thing that the node.js community has to do is to drive towards a framework that makes modern form-based applications dead simple to build.

What would make a framework based on node.js more mainstream?
Right now, modern webapps like Quora, Asana, Google Docs, Facebook, Twitter, and others are setting the bar high for sites that can reflect changes in data across multiple users in real-time. However, building a site like this in Rails is extremely cumbersome in many ways that the node.js community may be able to solve more fundamentally.

That’s why I’d love to see a “Build a blog engine in 15 minutes with node.js” that proves that node could become the best way to build modern form-based applications in the future. In order to do this, I think you’d have to show:

• Baseline functionality around scaffolding that makes it as easy as Rails
• Real-time updates for comment counts, title changes, etc that automatically show across any viewers of the blog
• Collaborative editing of a single blog post
• Dead simple implementation of a real-time feed driving the site’s homepage

All of the above features are super annoying to implement in Rails, yet could be easy to do in node. It would be a huge improvement.

Until then, I think people will still continue to mostly build in Rails with a large contingent going to iOS – the latter not due to the superiority of the development platform, but rather because that’s what is needed to access iOS users.

UPDATE: I just saw Meteor on Hacker News which looks promising. Very cool.

Will CPE (Cost Per Engagement) advertising ever take off?
I doubt it – the reason is that it’s targeting metrics at the kind of marketers that don’t care too much about metrics.

Broadly speaking, there’s two kind of marketers in the world – a ton could be written about this, so I’ll just provide some sweeping generalizations:

Direct response marketers are companies that are typically very focused on ROI when they buy advertising – often these include companies you’ve never heard of in ecommerce, online dating, financial services, etc., where it’s easy to calculate the value of a customer and they are primarily getting their traffic through paid marketing channels. They like to back everything out to ROI by comparing lifetime value to cost per customer, and if not that, then at least cost-per-action or some similarly concrete metric.

In many cases, these kinds of marketers prefer search marketing, email marketing, telesales, and other things where it’s easy to quantify what’s going on – they stay away from Super Bowl ads though. They prefer CPA and CPC versus CPM or sponsorships.

Brand marketers are companies you’ve heard of and have seen a lot of advertising for – they are typically targeting a large consumer base, they want to position their products differently relative to their competition and don’t have great ways to quantify the value of a customer. For example, Coca-Cola doesn’t know the LTV of a customer nor what the cost-per-customer looks like for a billboard ad they’ve bought.

For these guys, they are used to hiring big ad agencies to help them advertise on billboards, television, the front page of Yahoo, etc. They may buy search marketing, but have different goals than ROI. (For example, they may just want the top ad, and don’t care too much about ROI)

Why CPE is a weird metric for both DR and brands
The reason why cost-per-engagement is a weird metric is that ROI-focused marketers (that is, direct response marketers), don’t care about “engagement.” They want to know if people are going to buy, and if their media spend is going to be profitable.

As a result, the “E” part of CPE is really only a part that brands care about. And yet, they don’t care that much about CPE because they aren’t focused on the cost of the campaign as the #1 priority. Instead, it’s more important where the ads are being placed, how strong the ad creative is being used, etc.

One scenario to demonstrate this: If they could buy the front page of YouTube, even if that had a higher CPE, a brand advertiser would be happier with that than being shown in random footers of YouTube (the “remnant”) even at a lower CPE. They are looking to establish their brand, not optimize their spend.

What will be prevalent instead?
I think even with the advent of lots of ad opportunities on social sites, the dominate business model will still be CPM/sponsorships for brand advertisers, and CPC/CPA for direct response. Basically, nothing much will change.

If it turns out that CPE correlates to CPA/CPC, then DR marketers will end up liking it.

Also, CPE might turn into a secondary metric that you use alongside really strong placement of ads- maybe as a way to establish a bonus or upside on the campaign, but I don’t think it’ll ever happen that the dominant form of advertising on the web will be that ad agencies will put in a CPE “bid” into self-serve systems :)

I answered this question on Quora – more great answers over there.

Facebook, early 2006
Sometimes, you need to be horribly, embarrassingly wrong to remind yourself to keep an open mind. This is my story of my failure to understand Facebook’s potential.

In 2006, I was working on a new ad network business that experimented a lot with targeting ads with social network data, broadly known as “retargeting” now. The idea was that we’d be able to take your interests and target advertising towards them, which would lead to higher CPMs. As part of this project, we did a meeting with Facebook when they were ~12 people. I had read bits and pieces about the company in the news, but since I was a few years out of college, I hadn’t used their product much. We got a meeting and since I was based in Seattle at the time, I flew down with some coworkers and chatted with them at their new office in Palo Alto.

We met the Facebook team at their office right next to the Sushitomo on University Ave. The place looked like a frat house – a TV and video game console on the ground, clothes and trash everywhere – the result of a handful of young people working very hard. After waiting a few minutes, we were escorted into a meeting room where we met with Sean Parker, Matt Cohler, and Mark Zuckerberg. Sean led the meeting, and told us a lot about Facebook, the amazing job he did raising their recent VC round from Accel, and all the good things that were happening at the company. Mark and the other folks there didn’t say a thing.

Ultimately, we didn’t get to work with them though we did eventually sign 1000s of publishers including MySpace, AOL, Wall St. Journal, NY Times, and others. But that meeting opened my eyes and convinced me of a horribly wrong thing: Facebook would never be a billion dollar company.

The metrics for Facebook – high growth, very low CPMs
As part of our meeting, we talked a bit about the metrics around Facebook, and I was immediately struck by a few things:

• Facebook was growing fast- very fast, and impressively handled by a super young team (like me!) sitting on a site with millions of uniques/month
• Their CPMs were terrible, lower than $0.25 (the revenue earned per thousand ad impressions) and the site was covered (at the time) with crappy remnant ads like online poker, dating, mortgages, etc. (ironically, which we now associate with MySpace)
• They didn’t know much about advertising, and that their CPMs were really bad and unlikely to improve- their monetization strategy seemed superficial at best

From these numbers, I did a quick calculation:

$0.25 CPM * 5 billion ad impressions per month max?
= $1.25M/month = $15M/year = $150-300M value business?

I figured that Facebook hitting 5B ads/month would be incredible – after all, it was just a college social network, right? Hitting 5B impressions/month would make Facebook bigger than our largest client at the time, ESPN.com, a top 10 internet property. The only thing larger were big portals like Yahoo, MSN, and AOL. The idea that Facebook would one day be bigger than all the portals never crossed my mind.

I was confident especially in the CPM number staying low because I had multiple proprietary datapoints from across the industry – from MySpace, Friendster, Hi5, Dogster, and many other social networks. I was convinced that I had a unique understanding about Facebook’s true potential – that convinced me even more that it could never be big.

And of course, I was totally, horribly wrong :)

The case at Yahoo for buying Facebook
While I was doing these calculations after my meeting, Yahoo was also doing a similar analysis on the value of Facebook for their ill-fated attempt at buying the company. I would first read about it in the WSJ, but later saw this fascinating slide on Techcrunch.

The slide below starts out with a projection of how many registered users Facebook had at the time and projected very logically what it would mean for them to saturate more of the core userbase of “high school and young adult” – I’m sure at the time, these felt like aggressive projections to ultimately be able to justify a big purchase price:

If you look at these numbers and compare them to what really happened, it’s pretty hilarious. Comparing their projected 2010E and what actually happened, they were only off by a few hundred million users!

Furthermore, I would say that even the Yahoo numbers were very optimistic about the increase from CPM going from $0.25 to >$5 over time. There were a lot of problems with brand advertisers putting themselves next to user-generated content that had not been worked out, and these numbers would have also ultimately involved Facebook doing homepage takeovers and such. And in fact, it’s true that no large user-generated content or social networking site has been able to generate CPMs close to the $5 level, at scale.

So what was wrong with my reasoning?
Ultimately, all my conclusions were wrong by several orders of magnitude – Facebook would go on to become the #1 site on the internet and would break all attempts at reasoning based on historical datapoints, interpolation, expert opinions, etc.

To contrast how silly my reasoning turned out to be:

My 2006 prediction: Facebook would max out at 3-5B pageviews/month
Reality: Facebook is at 1 trillion pageviews/month, and growing

I was ultimately right on the CPMs not improving by much, but it didn’t matter because I was off by 200-300x on pageviews/month! Total fail. The big insight, of course, was that Facebook wouldn’t just stay a social network for college students – ultimately the product targeted the market of everyone in the world. Confined within this the college niche, the idea that Facebook would one day reach a trillion pageviews per month seemed ludicrous. But because of the vision of the founding team, Facebook broke through this niche to build a new product that the world had never seen, and got to the numbers I had never predicted.

The most exceptional cases defy simple pattern-matching
As I mentioned in my previous post on group think vs innovation in Silicon Valley, there’s a strange contradiction between the mental tools we use to analyze and categorize businesses versus what it looks like when there’s an exceptional company that takes off. Pattern matching, deductive reasoning, and expert opinion tell you how things work in the “typical” case, but of course, we’re not interested in the typical case – we’re trying to find the exceptional ones, the rocketship companies that define the startup landscape.

That’s exactly when our logical reasoning and historically-based reasoning fails us the most.

For example, after years of failures from the entire category of social shopping sites like ThisNext, Kaboodle, and others, Pinterest has become the hottest company of the year. After years of Google impressing upon all of us that every startup needed to have an algorithm called X-rank and a 10X technology advantage, a simplistic webapp known as Twitter would emerge. And after 10 VC-funded search companies were started, and people at Yahoo thought search was a loss-leading feature that would best be outsourced, Google emerged. The list goes on and on.

Legendary VC Mike Moritz, who invested in Google/Yahoo/PayPal/Apple/etc has a relevant quote here:

I rarely think about big themes. The business is like bird spotting. I don’t try to pick out the flock. Each one is different and I try to find an interestingly complected bird in a flock rather than try to make an observation about an entire flock. For that reason, while other firms may avoid companies because they perceive a certain investment sector as being overplayed or already mature, Sequoia is “careful not to redline neighborhoods.

There’s a lot to be said for investing in the ugly duckling. When Don Valentine led Sequoia Capital’s investment in Cisco, many others had passed on the husband and wife founding team of Len Bosack and Sandy Lerner.

Never has a more profound thing been said about birdspotting :)

The biggest lesson I took away
The concrete lesson to be learned from this is: In the modern era, business models are a commodity. I never want to hear about people asking, “But what’s their business model?” because in a world where you can grow a userbase of 1 billion in a few years, displaying remnant ads and getting a $0.25 CPM will do. Or just throw some freemium model on it, and monetize 1% of them. If you can build the audience, you can build a big business.

The more abstract lesson to learn is: Be humble, and keep an open mind towards weird new companies. After a few years in Silicon Valley, you can gather a lot of useful heuristics about what’s worked and what doesn’t work. That will help you most of the time, but when it comes to the exceptional cases, all bets are off. So keep your mind open to weird, young companies that you meet that don’t fit the established pattern: Maybe the founders will all be recent MBAs, or be a spinout from a stodgy old corporation. Or maybe it’ll be in a slow-moving market, or it’ll be a married couple, or there’s 10 founders, or some other stereotypically bad thing. Remember that you’re helping/investing/working for the company right in front you, not a mutual fund of all companies with that characteristic!

If you had looked at social networking companies as a group, as I did, you would have found a flock of companies with questionable business models. However, if you had been prescient enough to pick out Facebook specifically, then you would have seen a company break through all historic precedents and become a huge success. Hats off to all 12 employees I met that day in 2006.

Once you’ve been working in Silicon Valley for a bit, you’re often offered advice such as:

• Are you launching at X conference?  … where X is whatever hot conference is coming up, like SXSW or Launch
• Do you have an X app? … where X is whatever new platform just emerged, be it Open Social, iPhone, or whatever
• Have you pitched X venture capitalist? … where X is a prolific headline-grabbing investor with a recent hot deal
• You should do feature X that company Y does! … where X is some sexy (but possibly superficial feature) that a hot startup has done
• Do you know what your X metric is? … where X is some metric a recent blog post was written about
• Have you met X? … where X is some highly connected expert in the field
• Maybe you should pivot into X space! … where X is a space with a hot company that just raised a ton of funding
• Did you think about applying framework X to this? … where X is a new framework, be it gamification or viral loops or Lean

Sound familiar? I confess that I’ve both received and given much advice along the lines of the above. I call it “advice autopilot.”

The perils of “advice autopilot”
Advice autopilot is when you’re too lazy to think originally about a problem, instead regurgitate whatever smart thing you read on Quora or Hacker News. If you’re a bit more connected, instead you might parrot back what’s being spoken at during Silicon Valley events and boardrooms, yet the activity is still the same – everyone gets the same advice, regardless of situation. The problem is, the best advice rarely comes in this kind of format – instead, the advice will start out with “it depends…” and takes into account an infinite array of contextual and situational things that aren’t obvious. However, we are all lazy and so instead we go on autopilot, and do, read, say, and build, all the same things.

That’s not to say that sometimes generic advice isn’t good advice – sometimes it is, especially for noob teams who are working off an incomplete set of knowledge. Often you may not have the answers, but the questions can lead to interesting conversations. You may not be able to say “you should do an iPhone app” but it’s definitely useful to ask, “how does mobile fit into this?” This can help a lot.

The other manifestation of this advice autopilot is the dreaded use of “pattern matching” to recommend solutions and actions.

Pattern-matching in a world of low probability, exceptional outcomes
One of Silicon Valley’s biggest contradictions is the love of two diametrically opposed things:

• The use of pattern-recognition to predict the future…
• … and the obsession with a small number of exceptional successes.

Exceptional outcomes for startups are limited – let’s say it’s really only 5-10 companies per year. In this group, you’d include companies like Facebook and Google that have “made it” and hit $100B valuations. On the emerging side, this would include startups who might ultimately have a shot at this, like Dropbox, Square, Airbnb, Twitter, etc. This is an extraordinarily small set of companies, and it isn’t much data.

The problem is, we’re hairless apes that like to recognize patterns, even in random noise. So as a result, we make little rules for ourselves – Entrepreneurs who are Harvard dropouts are good, but dropping out of Stanford grad school is even better. It’s good if they start a company in their 20s unless they’re Jeff Bezos. Being an alum of Google is good, but being an alum of Paypal is even better. Hardcore engineers as founders is good, but the list of exceptions is long: Airbnb, Pinterest, Zynga, Fab, and many others. And whatever you do, don’t fund husband-wife teams, unless they start VMWare or Cisco, in which case forget that piece of advice.

As anyone who’s taken a little statistics knows, when you have a small dataset and lots of variables, you can’t predict shit. And yet we try!

The intense focus on a small set of companies also introduces a well-known logical fallacy called Survivorship Bias. Here’s the Wikipedia page, it’s interesting reading. Basically, the idea is that we draw our pattern-recognition from well-publicized successful companies while ignoring the negative data from companies that might have done many of the same things, but end up with unpublicized failures. We’re all so intimately familiar with stories like “two PhDs from Stanford start Google” that we ignore all the cases where two PhDs from Stanford try to start a company and fail. Or similarly, YCombinator has built a great rep on companies like Airbnb and Dropbox, and yet you’d think that if you invest in 600+ startups that you’d get a few hits. Because of factors like this, it might seem as though A predicts B when in fact, it does nothing of the sort – we’re just not taking the entire dataset into account.

Conformity leads to average outcomes when we seek exceptional outcomes
The problem with giving and taking so much of the same advice is that ultimately it breeds conformity, which is another way if saying it reduces the variance in the outcome. And if you conform enough, you end up creating the average outcome:

The average outcome for entrepreneurs is, your startup fails.

Lets not forget that. And so one part of Naval Ravikant’s talk on fundable startups that resonated with me is the idea of playing to your extremes. He says in the talk:

“Investors are trying to find the exceptional outcomes, so they are looking for something exceptional about the company. Instead of trying to do everything well (traction, team, product, social proof, pitch, etc), do one thing exceptional. As a startup you have to be exceptional in at least one regard.” –Naval Ravikant @naval

Be extremely good at something, and invest in it disproportionately relative to your competition – this gives you the opportunity to actually create an extreme outcome. Otherwise, the average outcome doesn’t seem so good.

The flipside of innovation
The funny thing with all of this, of course, is that this is what innovation looks like. The remarkable ability for practical knowledge to disseminate amongst the Bay Area tech community is what makes it so strong. Before something becomes autopilot advice for a wide variety of people, often a small number of hard-working teams who know what they’re doing leverage it to great success. Follow those people, and you might find yourself successful – just like them.

So the billion dollar question is – how do you separate out trendy/junk advice from what really matters?

… well, it depends!

I recently dug through Favstar.fm and found a bunch of the tweets over the last few months that were saved/retweeted the most. Wanted to save them here for posterity:

Teardowns

• Facebook, Google, Twitter, eBay, YouTube, Wikipedia, Amazon, Hotmail, Blogger, Apple: How they used to look http://t.co/fL2zDHu0
• The Secret To Pinterest’s Astounding Success: A Brilliant Sign-Up Process You Should Copy http://t.co/AsGi9pBx
• Airbnb’s first pitch deck http://t.co/3BTSY6dO
• Android Gripes, Why do apps from the same company look worse on Android than on iPhone? http://bit.ly/h19EKL
• Why Angry Birds is so successful and popular: a cognitive teardown of the user experience http://bit.ly/dN3W3d

Compilations

• 11 Books Every Leader Should Read – Bob Sutton http://t.co/gwViu9DQ
• 5 Articles on Rapid Prototyping you Should Have Read – LaunchBit http://t.co/cF10LzKI
• Platforms and Networks: Managing Startups: Best Posts of 2011 http://t.co/mdeyv3wK
• 5 Former Design Trends That Aren’t Cool Anymore (So Stop Using Them) | Design Shack http://t.co/59prlt1N

Quotes

• “Success is like being pregnant: everybody congratulates you but nobody knows how many times you were fucked…” via @NatalieSEO
• “No matter how beautiful, no matter how cool your interface, it would be better if there were less of it.” –Alan Cooper (via @destraynor)
• Very appropriate for entrepreneurs: “A casual stroll through the lunatic asylum shows that faith does not prove anything.” -Nietzsche
• grid is the new feed, custom cover is the new custom background, real name is the new username, and repin/reblog is the new embed code.
• “The easiest way to get 1 million people paying is to get 1 billion people using” -Phil Libin, CEO Evernote http://bit.ly/f1SY7U

Media

• Steve Jobs – 25 years old, rare footage of him presenting about early Apple http://t.co/AiNbyW0S
• INFOGRAPHIC: How rich are the superrich? http://bit.ly/hxavXB
• INFOGRAPHIC: Carbs Are Killing You http://t.co/FMdZ5pEG
• Amazing PDF with lots of conversion %s to compare your product against http://bit.ly/e2t6Ip

Articles

• Bye Bye, Long Tail http://tcrn.ch/gfTOYP
• interesting lesson on listening to customer self-reporting: Walmart’s $1.85 billon dollar mistake – Daily Artifacts http://t.co/8GLLajdx
• Design is Horseshit! http://t.co/12vw2MZN
• If You’re Competing On Features You’ve Already Lost http://bit.ly/jbfk8p
• My answer on Quora to: What are the best metrics for measuring user engagement? http://qr.ae/OyAD
• i find myself explaining scalable startups vs smallbiz / lifestyle all the time. This blog breaks it down. Via @sgblank http://t.co/DJNSkTx
• Why Do Some People Learn Faster? | Wired Science | Wired.com http://t.co/Zd53Z3Bl
• The Top Ten Signs the Valley is on Tilt Again http://bit.ly/fjZUhk

Here’s the form as a link if you can’t see it – questions, thoughts, etc appreciated.

Lobby of the Pixar offices in Emeryville, CA

“My product isn’t quite there yet.”
You’ve said this before. We all have.

Anyone working on getting their first product out to market will often have the feeling that their product isn’t quite ready. Or even once it’s out and being used, nothing will seem as perfect as they could be, and if you only did X, Y, and Z, then it woould be a little better. In a functional case, this leads to a great roadmap of potential improvements, and in a dysfunctional case, it leads to unlaunched products that are endlessly iterated upon without a conclusion.

About a year ago I visited Pixar’s offices and learned a little about this product, and I wanted to share this small story below:

Over at Pixar…
Matt Silas (@matty8r), a long-time Pixar employee offered to take me on a tour of their offices and I accepted his gracious offer. After an hour-long drive from Palo Alto to Emeryville, Matt showed up while I was admiring a glass case full of Oscars, and started full tour. I didn’t take great photos, so here’s some better ones so you can see what it’s like: Venturebeat, Urbanpeak.

I’ve always been a huge fan of Pixar – not just their products, but also their process and culture. There’s a lot to say about Pixar and their utterly fascinating process for creating movies, and I’d hugely recommend this book: To Infinity and Beyond. It gave me a kick to know that Pixar uses some very collaborative and iterative methods for making their movies – after all, a lot of what they do is software. Here’s some quick examples:

• Pixar’s teams are ultimately a collaboration of creative people and software engineers. This is reflected at the very top by John Lasseter and Ed Catmull
• The process of coming up with a Pixar movie starts with the story – then the storyboard – then many other low-fidelity methods to prototype what they are ultimately make
• They have a daily “build” of their movies in progress so they know where they stand, with sketches and crappy CGI filling holes where needed – compare this to traditional moviemaking where it’s only at the end
• Sometimes, as with the original version of Toy Story, they have to stop doing what they’re doing and restart the entire moviemaking process since the whole thing isn’t clicking – sound familiar, right?

The other connection to the tech world is that Steve Jobs personally oversaw the design of their office space. Here’s a great little excerpt on this, from director Brad Bird (who directed The Incredibles):

“Then there’s our building. In the center, he created this big atrium area, which seems initially like a waste of space. The reason he did it was that everybody goes off and works in their individual areas. People who work on software code are here, people who animate are there, and people who do designs are over there. Steve put the mailboxes, the meetings rooms, the cafeteria, and, most insidiously and brilliantly, the bathrooms in the center—which initially drove us crazy—so that you run into everybody during the course of a day. [Jobs] realized that when people run into each other, when they make eye contact, things happen. So he made it impossible for you not to run into the rest of the company.”

Anyway, I heard a bunch of stories like this and more – and as expected, the tour was incredible, and near the end, we stopped at the Pixar gift shop.

There, I asked Matt a casual question that had an answer I remember well, a year later:

Me: “What’s your favorite Pixar movie?”
Matt: *SIGH*
Me: “Haha! Why the sigh?”
Matt: “This is such a tough question, because they are all good. And yet at the same time, it can be hard to watch one that you’ve worked on, because you spend so many hours on it. You know all the little choices you made, and all the shortcuts that were taken. And you remember the riskier things you could have tried but ended up not, because you couldn’t risk the schedule. And so when you are watching the movie, you can see all the flaws, and it isn’t until you see the faces of your friends and family that you start to forget them.”

Wow! So profound.

A company like Pixar, who undoubtedly produces some of the most beloved and polished experiences in the world, ultimately still cannot produce an outcome where everyone on the team thinks it is the best. And after thinking about why, the reason is obvious and simple – to have the foresight and the skill to refine something to the point of making it great also requires the ability to be hugely critical. More critical, I think, than your ability to even improve or resolve the design problems fast enough. And because design all comes to making a whole series of tradeoffs, ultimately you don’t end up having what you want.

The lesson: You’ll always be unhappy
What I took away from this conversation is that many of us working to make our products great will never be satisfied. A great man once said, your product is shit – and maybe you will always think it is. Yet at the same time, it is our creative struggle with what we do that ultimately makes our creations better and better. And one day, even if you still think your product stinks, you’ll watch a customer use it and become delighted.

And for a brief moment, you’ll forget what it is that you were unhappy about.

Special thanks to Matt Silas (@matty8r, follow him!) for giving me a unique experience at Pixar. (Finally, I leave you with a photo of me posing next to Luxo Jr.)

Quick blog post to congratulate my sister Ada Chen and her husband Sachin Rekhi, who have just announced the acquisition of their startup Connected to LinkedIn. The company was backed by a seed investment from Ignition Partners and Trinity Ventures.

Here’s an excerpt from AllThingsD:

LinkedIn has acquired Connected, a small contact management startups that unifies and dynamically updates users’ connections on email, social networks, calendars and phones, according to sources close to the company.

Connected is similar to Xobni/Smartr, but it’s more of a dashboard than a plug-in, and it costs $9.99 per month. The company had raised a seed round of $500,000 led by Trinity Ventures in June. The service has been called “bloody awesome” by Tim O’Reilly.

Ada and Sachin posted some additional info via blog post with the annoucement. They’ll soon be moving down to Mountain View as part of the purchase so I’ll get to see them more often!

Yesterday I attended the 500Startups demo day – it was a fun event and will be interesting to compare to the YC demo day coming up later this month as well.

For everyone who didn’t make it, I wanted to share all the slides:

Read the rest of this entry »

The “Ultimate Driving Machine” is a classic slogan that makes BMW compete based on position, not features.

It’s hard to keep things simple, especially when adding so many new features
In my recent post on the virtues of marketing simple products, a couple readers wrote in to write a really interesting questions – here’s a particularly interesting one by Mark Hull:

How do you ensure that by simplifying your product too much, you are not losing a competitive edge by a lack of additional features/functions?

Every product team struggles with this question- it seems like naturally adding more featureset adds more power to the product, yet at the same time adds complexity that makes it hard for new users to even get started. This is a common problem in the initial version of a product, because most of the time the first version doesn’t work, and the most obvious way to solve the problem is to just keep adding features until it starts to click. Yet does this ever work?

Don’t compete on features. If your core concept isn’t working, rework the description of the product rather than adding new stuff.

Make sure you’re creating a product that competes because it’s taking a fundamentally different position in the market. If the market is full of complex, enterprise tools, then make a simpler product aimed at individuals. If the market is made up of fancy, high-end wines, then create one that’s cheaper, younger, and more casual. If the market is full of long-form text blogging tools, then make one that makes it easy to communicate in 140 character bursts. If computers are techy and cheap, then make one that’s human and more premium. These ideas are not about features, these are fundamentally different positions in the market.

BMW is the Ultimate Driving Machine
My favorite example of differentiated market positioning in a very crowded market is BMW’s “Ultimate Driving Machine” slogan. It’s not just a marketing message, you know it’s true when you sit inside a BMW and turn on the engine. Among other things, you’ll notice that:

• The center console is aimed towards you, the driver
• The window controls are next to your stick so it’s easier for your right hand*
• … and obviously the remarkable driving experience

Furthermore, when you go to the dealership, the entire experience keeps reinforcing the “Ultimate Driving Machine” message. The point is, the positioning is about the driving experience and the engineering to back that up.

In a price and features comparison, it’s unlikely that BMW would ever come on top- it’s expensive, and very little of the money goes into the interior and niceties that you’d expect out of a Mercedes. Yet people end up buying BMWs not for the features, but because it’s a fundamentally different car than a Mercedes (or at least it feels that way).

I’ve always felt that Apple goes this way too, where their products are more expensive and often do a lot less than competitive devices, yet win because they have a more cohesive design intention across their whole UX. Again, the idea here is more about competing via a differentiated positioning rather than based on a feature checklist.

You’ll never win on features against a market leader
The other important part to remember is that for the most part, if there’s a winning product X on the market, you’re unlikely to win by creating the entire featureset of X+1 by adding more features. Here’s why:

• First off, that’s crazy because you have to build a fully featured product right away, and that might already take years to match a market leader
• Secondly, as described in the Innovator’s Dilemma, if you’re mostly copying the market leader and then adding features, those features are likely to be sustaining innovations that is likely on the incumbents roadmap already- by the time you’re done, they’ll either have it or just copy you

Instead, the idea is to have a simpler product that attacks the low-end of the market leader’s product by taking a completely different market positioning. That way, you don’t have to build a fully featured product and you can take a completely different design intention, which leads to a disruptive innovation.

Ramifications for startups building initial versions of a product
I think there are three key ramifications for teams building the first version of a product.

The first is: Don’t compete on features. Find an interesting way to position yourself differently – not better, just differently – than your competitors and build a small featureset that addresses that use case well. Then once you get a toehold in the market, you can figure out what to do there. This doesn’t mean that new features are inherently bad, of course- they are fine, as long as they support the differentiation that you’re promising.

The second thing is: If your product initially doesn’t find a fit in the market (as is common), don’t react by adding additional new features to “fix” the problem. That rarely works. Instead, rethink how you’re describing the product and how you deliver differentiated value in the first 30 seconds. Rework the core of the experience and build a roadmap of new features that reflects the differentiated positioning. Avoid add-ons.

The third is: Make sure your product reflects the market positioning- this isn’t just marketing you know! If your product is called the Ultimate Driving Machine, don’t just slap that onto your ads and call it a day. Instead, bring that positioning into the core of your product so that it’s immediately obvious to anyone using it- it’s only in that way your product will be fundamentally differentiated from the start.

* UPDATE: An astute reader, Greg Eoyang, pointed out that the modern generation BMWs (E90s) are different now- I have an E46 that’s a few years old, so I was basing my observation on that. He writes:

First of all, a most modern BMWs do not have the window controls near the stick, that’s like 2 generations old, they are on the windows just like Honda’s these days.  BMW doesn’t even tell you about a lot of the features that have been standard for a long time – such as speed variable volume on the radios – Wide Open Throttle switch (back in the non-CPU days, it cut off the air conditioner when you floored it) – They have improved the concept of a car which is more than the features.

Thanks for the additions Greg!

This is reposted from my answer on Quora here.

Question: What UX considerations were built into Google+?
The most interesting design choice I’ve seen for G+ has been deploying it across all the Google properties within a navbar, and via the notifications – I’m talking about this thing right here:

(btw, looking at it now, I notice it’s the same coloring scheme as Quora, hilarious)

Building G+ on top of pre-existing, high-retention products
Obviously this is a smart decision because it lets them build on top of their own high-retention, pre-existing products: Google Search and Gmail, in particular. Contrast this to an approach where they would have started up G+ as its own independent property, which Google users could choose to adopt or not- but then that looks like Orkut.

Anyway, as a result of adding this new global navbar across all the Google properties, they have to deal with a very small amount of real estate to create some pretty rich interactions. Thus, it was very interesting to then see them building a mobile-like interface for interacting with comments, follows, etc., inline, without leaving whatever experience you’re already in:

And if you click on any of these, you see a quick sliding motion that lets you interact with the different notifications inline, without going anywhere:

Contrasting with Facebook
In comparison, the Facebook notifications dropdown is almost more like a inbox of “pointers” to the actual content. As a result, while you can see what’s new, you can’t actually do anything about it without leaving where you are. I found this a nice interaction on G+’s part given that they are building on top of things like email or search where you may not want to leave yet.

Someone should obviously do a much longer design discussion of the G+ main site, but I personally found the new navbar and notifications system pretty interesting so I thought I’d write a bit about it.

Simple products aren’t only better designed, they’re easier to market too.
Marketing and product UX are seen as conflicting with one another, but there are, in fact, many opportunities for the two to work together. Some of the best tools for increasing metrics are the same ones that are used to create effective interaction designs. These techniques include things like adding “soft” onboarding experiences, stripping out unnecessary features, having clear visual hierarchy and calls to action, and many more tactics. Ultimately, these tactics serve to create simple product experiences that are both desirable and well-optimized.

Let’s explore the different reasons why simplicity is a virtue for both designers and marketing quants.

Highly optimized flows make it easy to understand “what do I do next?”
Every product lives and dies based on how well new users are able to sign up and get oriented with the product’s core value. High signup and onboarding rates depend on a large % of users completing each step.

As a result, it’s important for each page to be as simple and directed as possible, so it’s constantly obvious what to do next. If each page gives the user too many options, thus distracting from the primary goal of the funnel, then the %s will decrease. As a result, some of the best landing pages and funnels fundamentally depend on extremely simple, stripped down designs. Here, removing things like navigation chrome, extraneous links, etc is not only simpler, but also better performing from a metrics standpoint.

More data and faster learning cycles
A metrics-informed team depends on deploying A/B tests and evaluating the results as the core of their product iteration process. Early on however, you often don’t have enough users to quickly evaluate tests at a statistically significant level. This data is then further diluted when you have a complex featureset, since only a small % of users interact with each option. However, if you have a simple product, where almost 100% of the users go through the same signup, invite, and sharing flows, then you’ll be able to collect data sooner and thus make decisions faster too.

This is a huge advantage because when you can run A/B tests in 3 days instead of 9 days, for instance, you can learn 3x faster and find product breakthroughs sooner. Think about this like compounding interest in the bank- finding 10% improvements faster leads to exponentially better performance.

Simple products are easier to optimize and pivot
Ultimately, it’s the optimized flow through your product that wins – you don’t get any credit for complexity. One optimized funnel beats any number of unoptimized funnels, because you only get credit for average conversion rate across all the funnels. Thus, more funnels means that on a practical level, it’s harder to keep them all optimized. It’s easier and better to push users through a small number of signup flows that you can keep well-designed and well-optimized, so that the overall quality stays high.

This is especially true if you decide to make some product changes in a classic “pivot,” or otherwise test significant new additions in a signup funnel like adding Facebook sign-on. If you have a simple product with a small number of onboarding flows, then it’s easy to experiment to see if it’ll work, collect data quickly, and then add it to 100% of new users’ experiences. Contrast this to a complex product where shifting the design takes a lot of time because you have to update so many different places in the product.

Keeps the focus on top of funnel rather than low-impact add-ons

When a product isn’t working, often the knee-jerk response is to “fully bake” the product by adding more features. However, I’ve found that when examining the data of new startups, the problem most often lies on the first couple pages of a product- often an unattractive value proposition, or clunky signup flow that kills the new user experience. Adding metrics to simple products often makes it clear exactly what’s going on, and most of the time, it’s a fundamental issue that needs to be fixed on the first page.

In this way, simple products with the “right” value prop will end up with better signup rates- this lets you put your attention on top-of-funnel issues rather than low-impact feature add-ons that won’t 10x the destiny of your product.

Short funnels result in more conversions
One of the most powerful things you can do to a key product flow is to shorten it*. Generally, because you lose a % of users at each step, reducing the amount of work to get started is a highly effective tool- rather than presenting a complicated homepage and asking for tons of information upfront from a user, perhaps you just let them signup with Facebook- that might reduce the number of steps, leading to a simpler product and better metrics too.

Ultimately, this all aligns with the highly opinionated design ethos that prioritizes what users most often want to do, rather than presenting many options equally. As is discussed in the Palm story in the book “Designing Interactions” the features of a product are used in a Power Law distribution- a small number of features are used constantly and the rest are long tail. As a result, you want to make the most commonly used features convenient while putting the unused features available but hidden.

(*in some outliers, lengthening signup flows with the right steps can help too)

Increasing the prominence of high-value actions by removing low-value actions
One of the most common (bad) design patterns I see among metrics-oriented products is continually layering more and more prominent calls to action for sharing or other viral mechanics. This got especially bad in early Facebook apps. The problem is that the user’s attention is easily diluted, and each new feature competes with the last- as a result, after a few iterations of this, it’s pretty easy to end up with a frankenstein of a product that’s cluttered and messy.

Instead, a compelling tool is to remove features in order to make what remains more prominent. Instead of making the high-value actions bolded and highlighted in yellow, simply remove the actions that are no longer necessary. This leads to both a simpler product experience as well as raised prominence for whatever actions you want to emphasize.

Conclusion- let’s make design and metrics work together
Ultimately, the key to the tools above are that they increase the effectiveness of the UI while simultaneously increasing the metrics. This can happen because highly optimized products are dead simple to use- they have landing pages that communicate a compelling value, soft onboarding flows, clear calls to action, and simple mechanics that drive a lot of value. The same things that make it a highly marketable product are the same things that make it well-designed, and a great thing for which every product should strive.

To use these tools effectively, those who are metrics-informed must also become design-informed. While it’s obvious that you can increase the prominence of something by making it blink and highlighted in red, there are many more tasteful tools that lead to less visual clutter and provide an even greater metrics benefit. Even Dave McClure!

There’s more than one way to use this tool
A/B testing is a very useful tool that can be used to develop better product designs, rather than just evaluating landing pages.

In a classic A/B test, you’re metrics-driven and want to pick whatever test variant ends up with the higher numbers. This is a useful tool, but is only applicable to scenarios like signup flows where the conversion is obvious. This post will describe some different tactics that are metrics-informed and end up as an aid to your product design process, rather than driving it.

The tactics I’ll describe are for:

• Updating your product without negatively impacting numbers
• Streamlining your product by measuring and removing unused features
• Designing for the right level of prominence

Updating your product without negatively impacting numbers
Product teams are constantly pushing small updates to their products in response to customers and what’s happening to the market. When an update affects a key part of the product, particularly to the main signup flow or core viral loop, it’s often important to ensure that it doesn’t hurt the numbers.

For example, let’s say you’re building a new social site and you have a Facebook-integrated “friend finder” option that you want to add. If you build this and test it, you’ll likely find that since it’s unoptimized, it’ll have worse initial numbers. A classic A/B test will often eliminate the new design because it performs worse. But instead of killing it prematurely, you can use an A/B test to iteratively “bake” the new design with a small % of users until it’s ready to replace the old one.

If you know that it’s important to have this type of Facebook integration in your product design, what you do is leave it in, but only expose 10% of your users to it. Then keep making small updates to the design, working on the copy, call to action, and other aspects, until the new design performs as well as the original.

In this way, you can update your product without impacting the numbers negatively. And unlike a classic A/B test where you aim to just pick a winner, instead you are using it to incrementally benchmark a new design until it’s ready to replace the existing one. For this, you are design-led because you know you want to execute this product in a particular way, but you use the A/B test as a safety net to make sure you don’t push out something that’s not ready.

Streamlining your product by measuring feature usage
There’s an important design principle that says, “Do less, but better.” I’ll elaborate on my POV of this philosophy more in a future post, nevertheless many product teams struggle to remove features, or even to quantify unused features.

For example, you might have a legacy feature that suggests people to follow on your social site, which you’d like to replace with a Facebook-based “friend finder” screen instead. Sometimes it can be difficult to get rid of navigation on something like this because it’s not clear how many people are really using it and how that affects their behavior overall, especially new users

A nifty way of using A/B tests to handle this is to run an A/B test to remove the feature, and get the following information back:

• How many people actually get exposed to this feature? (Based on what % of people get added into the experiment versus your active users during the test’s time period)
• What metrics are affected by people who have this feature removed? (As long as the metrics are neutral to positive, then you can remove it safely)
• If some metrics are bad, can you counteract it by adding something else to the new design?

Similar to the process of updating your product, the important notion here is that you have a particular action you want to take on a design level (simplify the UX) and you use the A/B test as a tool to aid that design goal. In this case, rather than going with whatever has better metrics, instead the goal is to go with the better design as long as it’s neutral or better on the numbers.

Designing for the right level of prominence
As you model out the key metrics for your product, there’s often important assumptions that need to be made on things like what % of your users invite their friends, or how many friends they invite, etc. Oftentimes, entire product strategies hinge on making sure that certain kinds of metrics get hit- it could mean the difference between being a viral eyeballs business versus one based on lifetime value and ad spend.

From a product standpoint, this manifests itself as trying to figure out how prominent to make things like “Invite friends” or “Import your addressbook” or “Subscribe to the Pro version.” To build a great UX, you often want to make something as low-prominence as possible while still making sure it’s easy and accessible for users.

A/B testing can help a lot here since you can test multiple versions of prominence and see where it takes you. If you want to prove that a model is even possible (for example, in the very best case could we get 20% of our users to invite their friends?) then you can make a popup that asks for friend invites constantly and see if you are even close. The point here isn’t that you would ever actually close the experiment with the obnoxious popup, but rather, it helps you do a sensitivity analysis of what might even be possible, to see are realistic values within your model.

You can use this technique hand-in-hand with the other ones listed above so that you eventually take a high-prominence version of it and iterate until it’s acceptable to show to 100% of the users.

Final thoughts
The thing that all of these ideas share is that you are using A/B testing as a tool to aid in a broader and stronger design POV rather than slavishly following whatever has the better metrics outcome. As others have discussed before, it’s the difference between data-informed versus data-driven. Many features you’ll want to do in your product have lots of qualitative value, even if the short-term quantitative benefits are difficult to measure or not there at all- using these advanced tactics lets you continue to push out dramatic new designs but without hurting the metrics your business depends on.

Do you live outside of Silicon Valley? Watch this video
For all the startups and entrepreneurs outside of Silicon Valley, I want to direct you to this incredible summary by Naval Ravikant of AngelList and VentureHacks on the anatomy of a fundable startup.

This video is a much more comprehensive and detailed version of what I often talk to non-Valley entrepreneurs and startups about. It’s part of the “grooming” process that startups out here get to become fundable and to focus on the right things to get there. People spend a surprising amount of time on things that will contribute little or no value to getting them to a seed round, and this talk is the best I’ve seen in terms of presenting the issues in its entirety.

Naval broke down the 5 main qualities of an “exceptional startup,” in the following order:

1. Traction
2. Team
3. Product
4. Social Proof
5. Pitch/Presentation

And while all these qualities are important, Naval explained, the most important thing is to understand that:

“Investors are trying to find the exceptional outcomes, so they are looking for something exceptional about the company. Instead of trying to do everything well (traction, team, product, social proof, pitch, etc), do one thing exceptional. As a startup you have to be exceptional in at least one regard.”

(via Founder Institute)

Anyway, please watch it all the way through and enjoy!

7th Founder Showcase – Naval Ravikant Keynote from Founder Showcase on Vimeo.

Does anyone care about your new product?
A question every entrepreneur asks is, if I build it will they come?

You can have a cool idea for a new product, but how do you know if anyone cares about it? And for a consumer product, how do you know that tens of millions of people will care about it?

One of the key points that I argue in my 2011 blogging roadmap is that tapping into a large market with pre-existing demand makes things easier. More discussion here. This means going after markets where users are already familiar with with your product category, the different options, and you build a product that is better against some competitive axis.

This post is about one way to figure out if anyone cares about your product.

Innovating on product execution vs. Creating a new product category
Some of the Valley’s best companies, like Google, Facebook, and Apple’s mobile products, entered their respective markets late in the game and effectively competed with differentiated products to win. They competed by executing great products in pre-existing categories rather than creating brand new categories. At first I resisted this point of view, since it’s more fun to paint on a blank canvas and do something that is completely new and innovative. Yet over time, I’ve come to believe that “blank canvas” ideas may be superficially innovative, they are much riskier and “first mover advantage” is wildly overrated, especially when there’s ample room to innovate in existing product categories. And for ideas in spaces with lots of competition, if you are able to get to scale and differentiate along some key dimensions, ultimately your traction lets you do all sorts of fun innovative stuff later on.

It’s easier to reinvent something than to invent it.

The reference example of this is Apple, which has created amazing and differentiated products in huge pre-existing categories like computers, laptops, MP3 players, phones, etc., and only occasionally go for new product categories (like the Newton and iPad). When Apple picks an existing category, they can take something that’s OK but fragmented, and take it to an entirely new level on design- and they can do this without the risk that the market is zero.

So these days, as I meet new startups, I like to think about the new/risky stuff in their product as part of a careful and coherent strategy to tap into pre-existing markets, rather than trying to create new categories. One of the key tools that you can use here is the Google Keyword Tool.

Introducing the Google Keyword Tool
The GKT was created for advertisers looking to buy Adwords- in fact, some of the best market research tools on the internet are designed for ad buying, so that advertisers can actually understand how much inventory is available to buy. I’ll do similar writeups for Google’s Ad Planner tool, Quantcast, the Facebook ad buying UI, and others when I have time.

The GKT lets you do something very simple – Plug in some keywords, and it’ll tell you:

• How many searches are happening on those keywords
• Related keywords to what you put in

Here’s a screenshot:

You can use this for a lot of different scenarios, but my favorite use cases are to validate how mainstream a product category is, make sure you are using customer-centric wording to describe your product, and to identify nearby product positioning options.

Here’s an example of how I might use the tool to research a movies site:

1. Go to Google Keyword Tool
2. Plug in “movies” and sort by searches
3. Notice that some words, like “film” or “theater” are related, and add them to the search
4. Repeat, to collect a large collection of related keywords
5. Start scrolling through the results (again, sorted by searches)

Based on a GKT search like this, you find all sorts of interesting things, which let you both validate pre-existing demand and make sure you’re speaking the same language as your customers.

Validating pre-existing demand
First off, the # of searches is a pretty interesting. If you plug in keywords related to your business and find very low numbers, it might mean you’re using wording that mainstream users don’t understand or don’t care about. This happens commonly when a phrase is used to describe the product to other entrepreneurs and to investors, and includes abstract/strategic notions of what the product encompasses, but not what end-users are actually doing.

One practice I might suggest would be:

Steer your product towards a category with millions of pre-existing consumer searches – this shows mainstream understanding and demand for your product category

In fact, this is an easy way to define a new versus existing market that only applies to consumer internet products:

If people are searching for products in your category then you are in an existing market.

I personally find this a very nice and clear-cut way to figure out where you are is in the spectrum of new versus existing markets, and how much consumer behavior risk a product takes.

Are you using the words your customers use?
One of the best uses of the GKT is for finding the right words to describe your product. Oftentimes people like to use “X for Y” descriptions, which are convenient, and the Google Keyword Tool can help you refine that thinking.

If you plug in your high-concept pitch and you get millions of searches back, then you’re in good shape. For example, it turns out tens of millions of people are looking for “free movies,” so if you can do that legally, you’re all set. But sometimes you plug in a term and it falls flat. For example, imagine a startup that self-describes as “a marketplace for whatever” – in this case, “marketplace” is the X and the “whatever” is the Y. If you look up “marketplace” in GKT, what you’ll find that is that there are very few searches for that keyword, and there may be better options for the product positioning. Why does “marketplace” get so few searches?

My theory is that a “marketplace” is an abstract, businessy description for a startup’s strategy, whereas consumers likely only care about 1) selling stuff, or 2) buying stuff, and they are only in one mode at a time. As a result, I’d argue that as a marketplace startup might want to consider one of the following strategies for positioning their product:

• Targeting primarily one audience (buyers usually?) and have some secondary UI/flows to bring in sellers
• Picking a clearer attribute for what’s being listed, like “free” “collectible” or “upscale”
• Using colloquial terms “buy and sell” vs more abstract terms like “marketplace” or “exchange”

This product position then informs how you describe the product through all of your marketing channels. You’re probably better off buying ads or having site invites that say, “sell your useless junk online” rather than describing it as a marketplace.

Don’t build what your customers are asking for, says Henry Ford
An important reminder for this type of exercise, which is so dependent on what customers are searching for, is that what you build and how you describe it are two very different things.

There’s a famous Henry Ford quote:

If I asked my customers what they want, they simply would have said a faster horse.

This is absolutely true, and of course we’re all fortunate that Henry Ford ended up building a car. Yet at the very beginning, remember that they ended up calling cars “horseless carriages” so that the product could be anchored against something consumers already understood. Horseless carriage hints at many things – how it’s used and why it’s valuable, most importantly, and also the primary axis of differentiation. It’s horseless but still gets you from X to Y!

So maybe an addendum to the quote would be:

Build a car, not a faster horse, yet start by describing as a faster/better horse until people understand what cars are. Afterwards, build on that term.

Research potential segments of a market
Once you have a big juicy market to go after, the other interesting question is how you’re going to segment it. Basically, what is your product going to do that’s better/different than what’s already out there? Steve Blank has written a bunch of interesting content on this, including this post and this slide deck.

The Google Keyword Tool can also help with this – when you do a GKT search for a big “X” like “movies” you find all sorts of interesting modifiers to that phrase. Just glancing at the results, you see potential subcategories focused on:

• free movies
• imdb
• movie times
• movie quotes
• movie trailers

Beyond these, you’re starting to get to very few searches per month. Looking at this, if a startup had a really compelling product for one of the above scenarios, I’d certainly be really excited about them.

(One of the reasons I’m optimistic about the new YC startup Hellofax.com is knowing how many searches are online for easy and free email-to-fax services. Just search for “email” and you’ll see what I mean).

When you search for these keywords on Google.com, sometimes there’s already great products that cater specifically towards this group. For example, there are a ton of cheap airfare ticket sites. But sometimes, you find millions of users searching for something that doesn’t exist- that’s pretty awesome, and a great opportunity if you can execute a really compelling product there.

Google Keyword Tool product research checklist
If you already have a product in mind, here’s a quick list of things to think about:

• What’s your “high-concept pitch” for your product? (Often an “X for Y”)
• Are you using terminology that millions of consumers actually understand and know how to search for? Or do only other smart hackers or social media douchebags know what you’re talking about?
• Along what dimensions does your product compete with substitutes? (hopefully only 1 or 2 axes)
• Are there millions of consumers who care about that competitive axis?

And again, to repeat the observation re: the Henry Ford quote, have a vision for your product, and execute against that. Invent and build cars, but market it as a horseless carriage so that people know what they’re buying and why.

Is this too conservative?
Short answer: Yes, if followed too strictly.

Remember that the Google Keyword Tool test can validate an idea, but I’d be hard pressed to use it to invalidate an idea. Sometimes people will use things, and be passionate about it, but not search for it. Or maybe the product is mostly on mobile or Facebook and the searches are happening there, not Google. Either way, it’s a conservative tool, that’s to be sure.

I think there’s a spectrum of risk on introducing new products- the safest thing to do is to execute the hell out of a product design for a huge pre-existing product category and a competitive axis that people care about. Ideally you could validate that this market was already pre-existing and the competitive axis was an important one.

Yet most startups aren’t started this way, and instead take some risk in either picking a new category, or a market segmentation that’s driven by intuition. Ultimately I think it’s still important to try this out, when your entrepreneurial intuition says it’s the right way to go- but remember that you might have to go through a step of cleaning up the marketing and messaging around your product to slot it into something consumers understand.

Credit for inspiring this post
I have to give credit to Sean Ellis, who inspired me with a coffee conversation a few years ago to think about using search data to identify new versus existing markets. He was describing his consulting stints at Xobni versus LogMeIn as “demand creation” versus “demand harvesting” and using Google SEM as a test of which mode you’re in. Hopefully he will blog about that comparison with more nudging sometime :)

There’s been some excellent Groupon analysis
Since the S-1 has come out, there’s been some incredible analysis done – two of my favorites are Rakesh Agrawal’s Quora answer on “What are some notable aspects of the Groupon S-1?” and also Yipit’s analysis on the deterioration of fundamentals in Groupon’s oldest markets. I would highly encourage everyone doing anything in daily deals or the (misnamed) “social commerce” space to check those out. Additionally, please comment if there are some other great blog posts that I’m missing.

“There’s no tech! Anyone can do it!”
One of funny things that you used to hear about Groupon is how easy it is to start a clone, and how any startup could do it. A lot of people, especially developers, also say the same thing about product like Twitter, which are easy to code v1.0s for. That’s often a critique of consumer internet companies because what they do seems deceptively simple- there’s often no tech and no “barriers to entry” that a lot of the more B2B/enterprise investors like to see.

After all, let’s look at something like Groupon:

• Technology: Trivial, it’s just a mailing list and a landing page
• Market: Trivial to enter, because it’s huge and fragmented
• Sales: Trivial, you need 1 sales guy/gal initially who can sell some local deals

Seems like there ought to be tons of successful local daily deal sites right? And yet Groupon and Livingsocial control the vast majority of the market, and I have no idea who the #3 is? In fact, the most interesting competition ends up being other huge companies with big established userbases, like Yelp, Google, Facebook, Amazon, etc.

Email subscriber costs
The real reason is that there was a temporary arbitrage in buying tons of demographically targeted ad inventory that no longer exists. The Yipit blog post referenced earlier has this handy diagram:

That’s a huge increase from 2010 to 2011.

So if you think about it, this is one of the key bottlenecks to getting a Groupon clone actually started- if you want to build a list of 100,000 users, that’s actually going to cost you $3M right off the bat.

The backend is scary too
Furthermore, to even be able to monetize to break even, you start to need to contort the backend of your business to get there. This means that you have to be comfortable with things like:

• Extended time periods before your LTV catches up to your CAC (for example, 12 month breakeven on your LTV)
• Large # of deals per week at high margin
• To support the # of quality deals, a high-quality sales team

If you need 5 deals per week, every week, for 12 months to break even, then you’ll need a great sales team for that. Then you’ll need someone to optimize your ad spend, a bunch of customer support people, and all of a sudden it doesn’t look so easy.

Getting to scale, let’s say to 1M or 10M email subs, costs gobs of money that very few people in the world would be able to raise in venture capital. That’s why I imagine the most successful Groupon “clones” start in other geography where the arbitrage still looks like <$5 and not $30, or where it’s a high-end niche with some built-in distribution to get the first 10k-100k on board.

The same is true for viral products too
I wrote this post originally about Groupon, but it’s important to note that the same is true for viral marketing channels as well. As with other marketing vehicles, users get “inoculated” over time to the same approaches. Getting an “invite” was a big deal in 2003, so addressbook importers were super effective. Banner ads used to get 10% clickthrough rates, and now they’re 0.1%. Over time, marketing channels naturally become saturated and that creates a built-in defense against new entrants in the market.

Thus, even if something looks easy to build, you better do it quick otherwise you may never be able to catch up. A corollary to this is that if you discover a new marketing channel or some new viral mechanics, you’ll have a huge advantage early on since your response rates will be great.

Send me any other interesting analyses of their S-1 or others!
As all of these S-1s are coming out, I’ll try to stay on top of any interesting analyses, but feel free to email any that I might be missing. Just shoot me a note or comment on this post.

UPDATE: A post drilling down into how Groupon defines “customer” and ratios in their oldest markets (via Dru Wynings). Also, Yipit did a great followup post called “Reports of Groupon’s Death are Greatly Exaggerated“.

Today we have a guest post from my sister Ada Chen Rekhi about user acquisition based on experiences at her new startup Connected. Connected is a new contact management product they’re working on for professionals to easily manage their relationships across their email, calendar and social networks. Enjoy! -Andrew

When Does Paid Acquisition Work for SaaS Startups?
by Ada Chen Rekhi

Introduction
After recently moving on from adventures building a consumer gaming portal at Mochi Media (acquired last year for $80 MM), I’m now working on a new startup called Connected, which provides contact management without the work. I decided to blog some of my thoughts based on my experience thus far with deciding on the right user acquisition channels to focus on.

When does ad buying work for SaaS businesses?
It’s a convenient belief that after you decide to build your software as a service (SaaS), Google AdWords and other networks will enable you to outsource all of your marketing efforts and focus less about user acquisition. This is not always true. Here’s a “napkin math” model to quantitatively decide whether or not ad buying is right for your startup based on reality, not guesswork.

A model for user acquisition

Paid user acquisition works for you when the following proves true

• LTV > CAC

The lifetime value (LTV) of your users should exceed the cost of acquisition (CAC) to get them in the door. As a reminder

• LTV = Expected Life x Average Revenue Per User (ARPU) x Gross Margin

In addition, for SaaS, you care quite a bit about costs and conversion rate for your funnel to trial, and from trial to paid. In specific, these look like

• CPC – cost per click to get traffic
• % trial conversion rate – users who convert to a trial of your product
• % paid conversion rate – users who convert to paid account

To estimate your cost of acquisition, you can base it off of estimates for your trial and paid conversion rates.

• CAC = CPC / (% trial x % paid)

An example of cost of acquisition
Let’s pick an example and work backwards. Let’s say you have a

• $20/monthly subscription
• 5% paid conversion rate – from trial to paid
• 10% trial conversion rate – from visits to trial

Then let’s pick a two different points for cost per click

• $0.50 CPC
• $2.50 CPC

In order to get a user at these CPC points

• CAC = CPC / (% trial x % paid)
• CAC = $0.50 / (10% x 5%) = $100
• CAC = $2.50 / (10% x 5%) = $500

In this example, it costs anywhere from $100 to $500 to get a single paying user at $20 per month. If you were trying to acquire 100 users ($2000/month), at $0.50 CPC that’s $10k ad spend, and at $2.50 it’s $50k. Drew Houston from Dropbox brought up very similar issues from his Dropbox Startup Lessons Learned presentation, where their initial search marketing test had a whopping $233-388 cost per acquisition for a $99 product!

Compare this against lifetime value
Compare this against the lifetime value of your user, or the total amount of profit you expect to receive over the user’s use of your product. This value should factor in the churn that you’re seeing from users canceling their subscription over time as well as what the payback period and working capital which you expect. Even though you might expect a user to be retained over a period of years, most startups don’t have the capital necessary to tie up their money for that long.

Let’s go back to the example above. We have the two users who cost

• $100
• $500

Assuming zero churn and zero operating costs on their $20/month subscription, you would recoup your cost on these user over a fixed period of time

• $100 / $20 = 5 months
• $500 / $20 = 25 months

In the case of second user, it would take over two years to recoup the initial $500 you spent to acquire them. You can offset this issue of working capital by setting the value at the amount of revenue you receive over a fixed period of time, or by being more aggressive with pushing them to prepay for longer periods of subscription cost upfront.

For example, what if you could get these users to pre-purchase their $20/mon subscription for $149/year? You’d be able to recoup the first user’s cost instantaneously, and get back a significant percentage of the second user’s acquisition cost.

Making the model work
The path to achieving profitability looks like making the model of having your cost of acquisition beneath your lifetime value work. You can quickly get a back of the envelope idea of whether paid acquisition is for you based on the examples and model above.

Doing this will help you determine whether or not you can profitably use ad buying as a source for getting users. You can also fine-tune your model to incorporate even more granularity such as

• virality
• traffic source
• retention
• working capital
• churn
• etc.

Trying paid acquisition on for size

Now that we have the framework down, the question is whether or not paid acquisition works for you.

If this works for you, then congratulations- you are on the path to scalable riches! ;-) If it doesn’t work, then you should think about how far off it is. Getting ad arbitrage to work out profitably is extremely sensitive to changes in the steps of your conversion funnel, as well as the source of the traffic. So if you’re not many factors off, it may make sense to spend a few months refining your funnel and trying to optimize the channel the traffic is coming from. Here’s a few things to consider-

Does the math work?
Once you launch your product and get a sense of what the conversion rates are in each step along the funnel and the churn rate, it may be that the math doesn’t work out. If you’re not too far off, then it may be worth spending time trying to make the metrics work out through landing page optimization, increasing conversion along the steps of your funnel and trying to optimize your traffic sources. However, if you’re several factors off (this is common in highly competitive markets) paid acquisition may not make sense as a strategy for you.

Is your product in an existing market or a new market?
Intent-based paid acquisition channels like search advertising work best in an environment where users are aware of the problem and actively searching for solutions which your product meets. You can look up potential search terms and volumes through Google AdWords Traffic Estimator, including estimated average cost-per-click and monthly search volumes. If not, you can also experiment with targeting sites that reach the demographics of your users.

How much working capital do you have?
While theoretically you might be willing to pay up to the full LTV of the user, you may want to limit the amount you’re willing to pay based on a fixed time period, for example the expected value from the user over 6 months. This may be because at some point you run into working capital issues paying for users who may take years to break even.

I’ve had two recent conversations in which people have mentioned the “grow first, monetize later” philosophy as one of the signs of the coming bubble apocalypse, and this post is to argue why it’s very smart and rational to focus on getting millions of users first. (This post is part of my 2011 blogging roadmap)

Regarding monetization, I’ll note that…

• in general, consumer products mostly suck at monetizing
• any business model built on 1% subscriptions of 0.1% ads need millions of users
• costs are ridiculously low for new startups, and N millions of users is not expensive
• ignore this for products like marketplaces where monetization is part of the value prop

An ad-based example
Here’s some quick math- let’s say that you are a typical seed-stage team of 4 trying to get your startup off the ground, and your burn is approximately $40k/month. If you monetize at $0.25 CPM, which is a pretty typical ad rate for every thousand ad impressions, then that means you need a whopping 160M ad impressions per month to break even[1]. Even if you get 2X or 10X that ad rate, you’re still in the millions of users to get there. Scary right?

A subscription-based example
Similarly, if your site has a freemium business model, you’ll find that something like 1% of users subscribe for a pretty nicely tuned freemium configuration. So if you have 1% of registered users paying you $5/month, that means your average user is worth $0.05. Given this, you’d need 800k registered users, and if only 10% of your users register, you’ll need millions of users to get there.

Ultimately, the key is new user growth
Given the difficulty of monetization for consumer products, ultimately the best way to get to breakeven isn’t to try to optimize the 1% subscription rate to 2%, but rather to pick a huge market, create a killer product, and try to acquire millions of users. Because this is the biggest risk, you want to focus on growth first and foremost.

Here’s a different analogy that Steve Blank uses to get at this- let’s say that you wanted to create a cancer-curing drug. You don’t need to crunch the business model for that- if you had it, it’s valuable. You don’t need to price test or do customer development. All the risk is in the science, so you just focus on the science.

Similarly, I’d argue that in consumer internet, the real risk is that you can’t get millions of users actively engaged in your product, and that risk is ultimately driven by growth and long-term user retention. Thus focus on that first, then figure out the monetization once you’re at scale.

Stuff is so cheap these days
Note also that running a site with millions of users is cheap. The cost of hiring developers/designers will vastly overshadow the cost of maintaining the infrastructure- all you need are a few dedicated servers or just use Amazon Web Services- unlike the 90s, you don’t need a huge datacenter to get started. Because these costs are pretty low, you can just focus on making sure your designers and developers are productive and you’re getting to product/market fit.

Ignore this advice for products where revenue is part of the value prop
Of course for products where you are helping people make money as the central value, you need to do this sooner rather than later, so that you can make the entire network happen. So if you’re building a marketplace, collect money early, even if you don’t take very much profit. Same with Groupon-esque startups.

[1] CPM to revenue calculation
$0.25 CPM = $0.25 for every 1000 impressions
$0.25 / 1000 = $0.00025 per ad
$40k burn / $0.00025 per ad = 160M ad impressions per month