Andrew Chen Archives

This post is part of my recent 2011 blogging roadmap post, where I created an outline of going from zero to product/market fit. Getting to this endpoint is obviously a good goal in theory, but question is, what does it even mean to hit this goal?

The original definition
In Marc Andreessen’s original post on the topic, he writes:

Product/market fit means being in a good market with a product that can satisfy that market.

You can always feel when product/market fit isn’t happening. The customers aren’t quite getting value out of the product, word of mouth isn’t spreading, usage isn’t growing that fast, press reviews are kind of “blah”, the sales cycle takes too long, and lots of deals never close.

And you can always feel product/market fit when it’s happening. The customers are buying the product just as fast as you can make it — or usage is growing just as fast as you can add more servers. Money from customers is piling up in your company checking account. You’re hiring sales and customer support staff as fast as you can. Reporters are calling because they’ve heard about your hot new thing and they want to talk to you about it. You start getting entrepreneur of the year awards from Harvard Business School. Investment bankers are staking out your house. You could eat free for a year at Buck’s.

His partner, Ben Horowitz, follows it up with a bunch of other observations about the fact the event isn’t a “big bang” kind of event – instead, there’s lots of gray area as your product starts working for the market: I’d encourage everyone to read his subsequent post here.

So the short answer is, there’s no easy test.

Now given that caveat, I’m going look at this through the lens of consumer internet to add some additional thoughts.

What is a market anyway? And how do you validate it’s real?
How do you even define a market for consumer internet? Ultimately, I concluded that the most useful definition of “market” is 100% consumer-centric. Here’s an attempt at a simple definition, focused on consumer internet:

A market consists of all the consumers who can search for and compare products for a use case they already have in mind.

This definition is very focused on the notion of pre-existing demand for products in your market, and is scoped narrowly to avoid confusion.

The most concrete test of pre-existing demand is using the Google Keyword Tool, which tells you how many people are searching on Google for a particular keyword. To try this out, you’d execute the following steps:

1. What keyword do people search to get to your site?
2. Put those keywords into Google Keyword Tool
3. How many people are searching for this keyword?

If the answer to #3 is large (millions or more), then you have a large market. This test is very concrete, and also very finicky. By design, terms like “vacation package” score high on this test, whereas “travel experiences” do not, even though an educated entrepreneur or investor might abstractly group them together. Similarly, by design, a person who’s building a “social network for musicians” might be inclined to list the # of musicians in the US as part of their market sizing, but under this test, you’d quickly see that there’s not too many people are specifically looking for that. Also interestingly enough, you’d never say there was a “Photoshop market” but a quick search will show that in fact almost 40 million searches per month on “photoshop,” and it might be a great strategy to position yourself relative to that keyword.

Validating that you are part of a pre-existing market comes with all sorts of benefits, which I’ll address in later posts. But for now, the most important benefit is that you know the # of potential customers is large.

(In general, I’ve been constantly confused about how to even define a market in consumer internet, given that there’s so much similar featureset between otherwise very different products. For example, early on, people talked about “social” as if it were a type of site, whereas now it’s seen as an aspect for all new products coming to the web. Similarly, people sometimes talk about “Facebook apps” as if it’s a market when, again, it’ll probably just end up an aspect of every new online service.)

What’s a great market?
What are other attributes that make a market attractive? For consumer internet, a great market is commonly defined by:

• a large number of potential users
• high growth in # of potential users
• ease of user acquisition

Not competition, in my opinion, because for consumer internet there is often literally billions of potential users, and you’re mostly competing against obscurity. So even if there’s a ton of competition, if it’s easy to acquire consumers to your product, that’s great! Then get a good enough product, and you’re ready to go.

Not monetization, in my opinion,  because making money is pretty straightforward. You can throw on some ads and get $0.1-$1 CPMs, or you can charge subscription rates and get 1% to convert, or you can do the virtual goods thing. The biggest risk in all of these monetization models is really about whether or not you can get millions of users or not.

Picking a great market leads to better products
Leading with a great market helps you execute your product design in a simpler and cleaner way. The reason is that once you’ve picked a big market, you can take the time to figure out some user-centric attributes upon which to compete. This leads to a strong intention for your product design, which drives a clean and cohesive UX. In a market of all black Model Ts, you can sell otherwise identical cars of different color and that’ll work. Picking the right attribute is it’s own topic though!

The important part here is that you can usually pick some key things in which your product is different, but then default the rest of the product decisions. This means that your product’s design can be more cohesive because you’re trying to do less, but better.

Once you’ve executed your product, then there are various ways to validate that it’s “good enough” and your product fits the market:

• When user testing, do people group your product in with the “right” competitive products?
• Do they understand the differentiation of your product versus your competitors?
• Will some segment of users in the overall market switch to your product?
• Are some users who’ve “rejected” the products in the market willing to try your product?
• How do your underlying metrics (DAU/MAU, +1 week retention, etc.) compare to your competitors?

All of the above are signals towards product/market fit. Thee above tests are interesting in that they fundamentally anchored on pre-existing competitive products in the category. In a new market, you don’t have the luxury of comparing yourself to other things.

In future posts, I’ll try to give some more concrete metrics based on my research for what are good numbers in each of these cases, but for now, the important idea is just that in a large existing market you have more datapoints to at least say, “my product is at least as good as the other guy’s.”

New markets are a danger to good product design
In fact, one of the scariest things to me about new markets is that doing great product design for them is extremely hard. It’s so unconstrained that it’s hard to do anything other than add features, see what sticks, and iterate. This is fun except that keeping a cohesive product experience is quite hard, and removing features is usually harder than adding them. So at the end, you incur tons of product design debt that never gets paid off. (It’s not a surprise to me that Apple has a history of simplifying already successful product categories, rather than inventing brand new ones from scratch)

Conclusion
To summarize my main points in this essay, I’ve come to some simplifying definitions on how to validate product/market fit in consumer internet. For market, if you constrain the definition to people who know how to search for products in your category, you can develop a pretty concrete test evaluating pre-existing demand. And by leading with a market, you can develop a central design intention that leads to better product design. This in turn can then be validated by comparing your product metrics to competitor numbers, as well as user tests that focus on grouping and differentiation.

This leaves lots of unanswered questions, but hopefully is a start to my new blogging roadmap! More to come soon.

One important question that comes up all the time is, what makes a product easy to market?  I had a fun chat about the topic with Eric Florenzano and Eric Maguire who worked together at Mochi Media with my sister Ada. They also recently did YCombinator.

After the chat, eflo wrote up a helpful summary of some of the ideas we covered. I wanted to quickly share them with some comments:

1. Come up with one resounding use case–one thesis for how people should use the product.  Preferably this fits in with something that users already do and already understand.

I’m going to try to start blogging again!
It’s been a long time since I was in a good blogging rhythm, and I’m going to try to start doing it again :-) In preparation for this, I put together an outline of an output-driven set of milestones around product, that takes you from zero to a P/M fit product thats ready to scale on marketing/tech/etc.

As far as I can tell, this is all standard fare for companies in Silicon Valley. My desire to write these posts is ultimately about documenting what’s working for people and spreading the knowledge beyond Palo Alto, CA :-) All of these topics are ultimately derived by both my own projects as well as my advisory roles at venture-backed startups. (Some of these are listed here)

If you like the outline and want to stay up to date, just subscribe and follow me on Twitter.

Without further ado, here’s the outline- I hope to write at least a post or two per week:

Blogging roadmap goals

• “output-driven” roadmap for going from zero to product/market fit
• for small hackerish teams building consumer internet products
• the intention is to create a scalable startup that is going after a huge market, and generate huge returns for venture capital investors
• goal is to get to P/M fit in shortest time possible, defer everything else
  • defers monetization
  • defers marketing
  • defers scaling
  • (this is all by design)
• P/M fit takes a non-deterministic amount of time to get there, insanely hard, you’ll probably fail anyway
• the problem is 90% contextual, make up your own rules as you go

Concept prototype

Picking a product and market

• build for yourself (start with intuition)
• have a long-term vision
• base it off something that’s already big and already working
  • big makes it easy to test and collect feedback
  • already working means you have a good sense for minimum product
  • also, there’s pre-existing distribution channels as well
• figure out the options for competitive differentiation – this is the core design intention
  • talk to a lot of users, do a lot of research, compare a lot of products in the space
• dimensions for competitive differentiation
  • competitive dimensions
  • vertical audience
  • design intention
  • cheaper/niche
  • targeting rejectors
• validating that there’s LOTS of pre-existing “pull” for the market
  • search keywords
  • app leaderboards
• ideal goal: simple product with fundamentally different core design intention for large pre-existing market
  • bonus points for baked-in distribution, monetization, etc. but don’t let this lead the idea!!!
  • usually one killer feature (not a bunch of features)
• prototype: Landing page
  • what’s a good landing page experiment?
  • headlines, copywriting, hero shot, etc.
  • unique URLs
• anti-patterns:
  • “someone’s already done this” (desire for originality)
  • monetization/strategy-driven product ideas
  • technology in search of a market
  • “Wall Street” markets
  • lumping yourself into an aspirational market
  • comprehensive featureset done poorly

Paper/Wireframe prototype

Designing the initial product

• go for the minimum desirable product
  • might work :-)
  • the central design intention drives the product design
  • supports only the core use case, as minimum as possible
  • core UX should be 2-3 pages
  • limited functionality, done well. “Less but better”
  • Should build bare bone prototype in less than 2 weeks (really!)
  • flow-based product design
  • user quotes, then fill in with UI
• low-fidelity prototyping tools
  • easier and cheaper to make changes
  • fix defects earlier (Toyota lean manufacturing model)
  • engineers always want to prototype in code, but then sunk-cost fallacy
  • get feedback from people and iterate
• prototype: Core user flows, mocked up and ready to build
• anti-patterns:
  • “database-up” design
  • feature creep and low product self-esteem (v1 should look like a feature!)
  • comprehensive featureset all of it done poorly
  • lots of pet features that don’t fit into the core design intention

Code prototype

Coding the initial product

• build the prototype as fast as possible
• fill in any blanks left out of the prototype
• use the product yourself, iterate on it while keeping with the core design intention
• focus on key flows and prioritize over ancillary ones
• don’t worry about corner cases
• get it ready to be used by other people
• prototype: Live product, usable by other people
• anti-patterns:
  • taking too long
  • losing focus of the central design intention
  • not adjusting based on intuition and usage
  • overarchitecting, trying to make it scalable or modular or future-proofing in general

Friends and family alpha testing

• private beta goals
  • clean up core experience
  • make product usable over multiple visits
  • validate the core design intention
  • not scalable
• recruiting friends and family
  • focus on retention
  • are users coming back?
• recruiting random people
  • Find people from the existing market, rejectors, and outside the market
  • Learn from extreme users
  • Craigslist
  • Usertesting
• user testing
  • do they get it?
  • how would you describe this to a friend?
  • usability – remove the friction
  • would they switch? (for existing market users)
  • Net promotor score
• interpreting user feedback and learning to say “no”
  • which users fall into the target market? Hear them out
  • which users don’t? It’s OK (and maybe even good!) to have them reject
  • try not to add new features unless absolutely necessary
  • what features can you remove that aren’t part of the core?
• prototype: Simple product, polished by real use
• anti-patterns
  • Delusion- it’s not working but you think it is
  • Melancholy from user testing
  • Adding features without interpreting
  • Adding features that violate core design intention
  • Listening to out-of-market users
• is it working?
  • people understand the product
  • some subset of your users like it and use it
  • you like it :-)

Random people beta testing

• traffic testing goals
  • start polishing your onboarding flow
  • develop options for distribution
  • build some basic stats infrastructure
  • not meant to be scalable
• User acquisition tactics
  • ads
  • PR + launch page + slow stream
  • partnerships
  • power through it
• Collecting feedback
  • surveys
  • help and problems
  • recruit users to talk to
• prototype: Spreadsheet for signup flow, more polished signup flow
• is it working?
  • signups are happening
  • people are going through the core flow
  • retention/recurring usage from target users
  • product still works for you, and your friends/family

User flow optimization

• model your usage and figure out your core drivers
  • this is completely product specific
  • two examples- daily deal versus a chat site
  • whats your “metric of love?”
• prototype your funnel – explore!
  • flow chart
  • excel
  • SQL
  • formalize/finalize with dashboards
• identify major bottlenecks for why the product’s not working
  • start at the beginning of the flow
  • fix bottlenecks with A/B tests
• is it working?
  • how do the metrics compare to the usage model?
  • 10% signup
  • +1 day retention and +1 week retention
  • DAU/MAU
• anti-patterns:
  • trying to fix problems in core UX when signup is the problem
  • over-architecting stats infrastructure
  • trying to use a generic analytics product to answer situational questions

Ready to scale?

• Hopefully the major checkboxes are checked – at this point you’d have:
  • Huge market
  • Differentiated product
  • Product makes sense to normal people
  • Product is working for IRL people
  • Product is working for non-IRL people
  • Well-understood and optimized user flows
  • Ready to scale up
• Non-scaleable marketing, tech, and otherwise- that’s fine
• Now scale everything else :-)

Crisis, terror, and melancholy

• Is it good enough?
• Nobody likes my product!
• My product is a mess!
• It’s taking too long!
• Investors hate my product!
• I’m iterating in circles!
• When to work on a completely new idea?
• Iterations are getting diminishing returns and people still don’t love the product

Final note: Thanks to my friends who helped review and add to this: Vinnie at Yipit, Alex at Penzu, Rob Fitzpatrick, Kevin at Hyperink, Jamie/Justin at Mocospace, Ada/Sachin at Connected, Noah at Appsumo, Jason at Kima, and the other folks who helped

Joshua’s slides on Metrics-Driven Design got tweeted out during SXSW and I wanted to share them.

In general, I think all of the various MVP/customer-development oriented startups out there are struggling with how to incorporate design into their product process. And at the same time, more designery teams are trying to figure out how to get more agile. It’s hard. As someone smarter than me has observed, the vast majority of the MVP-oriented companies end up with pretty uninspired, incoherent products- and they don’t seem to get any better over time. So I think it’s a great challenge for the whole community to get more informed about design and figure out how to really make it work.

Great Google color-testing followup
In particular, in the first few slides there’s a really funny followup to Doug Bowman’s complaints about Google testing shades of blue. These slides claim that in fact, the color choice really did matter, and quite a bit so, and quotes Bing search guy saying that the decision was actually worth $80M. I suppose in retrospect it’s not surprising, because the bluer something is, the more it looks like a link- so given the visual signal, it is meaningful for users over billions of searches.

Metrics-informed versus metrics-driven
All that said, I do have to say that I much prefer the term “metrics-INFORMED design” rather than “metrics-driven.” You should really be driven based on your vision of the product and where you want it to go, not the metrics that you use to validate or learn about your vision. (I first read the distinction of being data-informed over data-driven in a talk by some Facebook product folks, and have much preferred it ever since – this topic probably deserves an entire post by itself).

Finally, the slides
Anyway, the Joshua’s slides are excellent and I’d encourage you to flip through them. The official place to read the details around this presentation is here, on his site. His Twitter is here.

I’ve been struggling with trying to blog more often yet being in the mood to write the kinds of posts that I do. Quora helps a little bit, but I find myself mostly just tweeting instead :-)

Anyway, I have a question for the readers of this blog- please vote and/or leave any comments on what you’d prefer to see on here:

I wrote this on Quora a while back, but forgot to cross-pollinate it on my blog, so apologies if you’re seeing this twice. As those who have been following this blog know, I had a great deal of respect for MySpace back in the early days and worked with the initial team back when the site was just a few million members- I’ve written about it here and here.

Anyway, here’s the question…

How did MySpace, with a smart team of people, do such a bad UI/UX job with the new design?

The answer’s simple:
In the new redesign, MySpace prioritized short-term monetization ahead of user experience due to its failing business fundamentals.

First off- let me state that I think the new MySpace is actually better than the old one. However the new MySpace is still not good enough, obviously, to turn around the product.

I recently spoke to an interaction designer who worked on the new MySpace, who told me an anecdote that blew my mind:

When the team was working on the new feed at the heart of MySpace, the interaction designers wanted to make bigger images so that it’d be easy to see what users’ friends were doing. Similarly, they wanted to make the feed more easily scannable and have more content per page on the feed. Basically, to turn the feed into a modern implementation the way Facebook, Twitter, Quora, and many others have set up.

However, they were aware that if they did this, then users would be less likely to click through to the images and thus would decrease pageviews. Given MySpace’s declining revenues, the interaction designers there were asked to actively design with the goal of more pageviews. So they added smaller images than they thought optimal, and fewer images per page than they thought optimal, just so that they could generate more pageviews. Basically they were now designing a worse newsfeed to generate short-term revenue.

As I understand, this happened systematically within the product which led to many compromises in the user experience, and the business needs won every time.

When the folks who ought to be the strongest user advocates at the company design for the business goals as a priority, you do not end up with an inspired product experience.

You have to prioritize having a great product experience to end up with a great product experience- it doesn’t happen by accident.

Anyway, the site is still huge and influential in many ways, so let’s hope the team there figures it out and there’s a resurgence in the future.

[ed: I also wanted to add the following answer from Sizhao Zao Yang, co-founder of myminilife, which created Farmville and then was acquired by Zynga]

Sizhao’s additional commentary on this question:

In addition to Andrew Chen‘s comment, I want to emphasize MySpace’s short term perspective seeped into product/engineering such that management actually believed that MySpace was special because people liked to generate pageviews.

MySpace invited a number of the application developers to MySpace last year including Zynga, and I was the Zynga representative. During the sessions, they asked for specific suggestions on product. I told them to make it feed based: increase the size of the pictures, have more descriptions on the activity, etc. Have different ways to surface social content with counters/toasters, and make social feedback very easy with one click functionality. Multiple general managers and product people at MySpace told me that MySpace people just like to click more. I told them that they were on “the wrong side of history.” Little did I know that this session was broadcast to all of MySpace. So, overall, my comments to the management/product ppl/everyone didn’t resonate at all, and most of it was never incorporated in the MySpace 2.0 launch. They said my comments were “interesting, but we’ll see,” with an underlying mood/attitude that I was wrong about the pageviews generating MySpace crowd.

I remember also that when FB was on the rise, MySpace execs would publicly say they were: cooler, more about self expression, celebrities, and that the newsfeed/app platform didn’t matter (in the early days) because it was too geeky. They didn’t know what was going on and positioned MySpace as brand and used lifestyle marketing to promote MySpace. Ultimately, you can probably blame the non- product focused culture, or you can blame the completely wrong judgment/perspective. They just didn’t get it. In b-school/MBA talk, these were strategic/product mistakes and a focus on the wrong metrics.

Unfortunately, when a company is in a downward spiral and think they are differentiating by encouraging pageviews, there’s not a lot you can do to help them. At the end of the day, you sometimes either get it or you don’t, unfortunately, MySpace went viral but didn’t understand social, which is about retention, not customer acquisition, and FB completely out executed/maneuvered them.

I posted this answer to Quora and figured I would share it here as well. You can find me on Quora here.

What are the best metrics for measuring user engagement?
Metrics are merely a reflection of the product strategy that you have in place.

What you are trying to do should lead what you want to measure, not the other way around. It’s for this reason that the blanket questions and answers around “best” metrics are meaningless- the question is, what are you trying to do.

For example, if you are an ecard that is driven based on holiday traffic, your strategy might be:

• people should come to my site at every major holiday
• people should send as many ecards as possible

In this case, your week-to-week retention isn’t important. The only question is whether or not you are sending out ecards, and whether or not you’re a new user or if you came back last holiday.

On the other hand, if you are trying to be more of a communications product, then you might want something like:

• people should come back for short durations every day to check their messages
• people should write messages occasionally, but mostly read a bunch of messages

In this case, you care a lot about DAUs and +1 day and +1 week retention. You might also put in a qualifier to that to make sure that people are actually reading/writing messages and not just showing up to an empty messages area.

So ultimately, the important part is to figure out what you are trying to do and what the expected behavior is around it. Only once you have that should you then ask yourself how you’d validate and test it using metrics.

Silicon Valley is mean to MBAs
This tumblr, Whartonite Seeks Code Monkey, made me laugh.

It’s full of emails from clueless Wharton MBAs which read like this:

LOL right?

This also reminds me of the famous quote on valuing startups:

Add $1,000,000 in value for every engineer.
Subtract $500,000 in value for every MBA.

Here’s why it’s hard: The nerd perspective is, they don’t need you
Much of the reason why it’s insanely hard to find a really good technical cofounder is that the best ones really don’t need you. Or at least they don’t think they need you.

Because there’s an illustrious track record of engineering-founded companies succeeding, spanning from HP to Facebook, there’s a lot of datapoints that say that a 20-yo Stanford computer science major can do it himself, or at least with his other CS roommates. Similarly, the very best alums out of places like Facebook and Google have lots of access to capital, advice, and people- these are all recipes for making you (the biz founder) completely irrelevant.

So I think the right point of view is just to accept that the amount of leverage strong technical folks in the Valley have is just the facts, and you’ll have to work around that.

Remember this:

They are not the code monkey. You are the biz monkey.

That’s just how it is.

Picking the right idea
One key way to mitigate this is to pick the right idea that doesn’t require ridiculous amounts of technical expertise upfront. You can build a great company that’s extremely sales driven rather than product driven in categories like:

• Enterprise sales
• Groupon for X
• Blog/media sites aka Content farms
• Marketplaces
• Ad network

I’m sure I’m leaving many other categories out.

For anything above, a lot of the work is in sales, and the actual technical infrastructure doesn’t require a strong engineer to pull together, at least initially. You’ll need them to scale it, but at that point hopefully you’ll have more money and more momentum.

For the kinds of ideas above, they might be easy enough to build in the short-run that you can get a different kind of coder at first. You can get someone who can code up a site and potentially have some visual design background, rather than an “engineer” who has theoretical understanding of computer science, understands performance tradeoffs, etc. There are more of the former than that latter in the world.

At the same time, note that many of the ideas above may not be particularly exciting to an engineer that wants to play with technologies. So perhaps something that combines the two can help – for example, MySQL is a great example of a cool technology (at the time) but clearly couldn’t have been turned into a company without a lot of business types running around.

Understanding and communicating what you really bring to the table
If you read through the Wharonite Seeks Code Monkey blog,  you can see that obviously they are mostly noobs and don’t know what exactly is the valuable part of what a biz cofounder can do versus not. This is true of many startups, both biz and geek-led, but there is huge overvaluation of the initial idea.

What do geeks really need help with? It’s very simple- there’s a class of purely business-related stuff that adds value:

• selling stuff and making money
• getting partnerships and marketing/distribution of the product
• funding the company
• scalable marketing/monetization strategy (ad arb / viral / freemium / etc.)
• team recruiting, particularly of other engineers and disciplines (not other MBAs please)

If you are an expert at any of the above and can show it, then there’s a lot more value. Very few business folks, particularly newly-minted MBAs (with the exception of Stanford folks) or industry-switchers can really deliver on these though, which is why they’re not bringing much to the table.

Then there’s a class of things that are much more product-oriented, and while it overlaps with the skillset of some engineers, if you have great skills in any of the following, they are clearly valuable too:

• design, especially visual design
• UI/frontend skills – HTML/CSS/JS – even if mediocre!
• copywriting within the product for help text, marketing, etc
• user research and customer development
• usability testing

Again, it all depends on what you’re really good at and what the particular product needs – enterprise might require less of the above, but a more solid initial product might help.

Worst comes to worst, write it yourself
And finally, there’s a nice track record of technical-enough people writing the first version of something and then having great engineers build it up later. Foursquare was like that, for example. More recently, David Binetti of Votizen wrote the first version of his product. I have immense respect for folks who do this, because it means they’re making “good-enough” progress without waiting for exactly the right technical partner to show up.

Any other thoughts or tips to share?
If you guys have other thoughts on ideas or thoughts on this topic, especially from those who are on the technical side, of how to attract and partner with engineers, write me a note in the comments! I’ll update this post as we go.

Per my previous post on Quora’s stats via the Facebook interface, I wanted to also share another hot startup at the moment, Instagram. Obviously you can use Instagram without Facebook- for example, only connecting it to Twitter, but again it shows the relative growth. In this case, as a mobile app, there’s very little data about how many folks are using it. Facebook’s data gives us unprecedented detail.

Here are the graphs, from AllFacebook – you can see they’re on a nice growth curve and doing quite well:

I ♥ Quora. Oh yes, I’m a fanboy.
As my many of my friends and family know, I love Quora and get a ton of value out of it. It’s incredible to see what some of the most intelligent and influential folks in Silicon Valley and beyond have to say, and it’s some of the most valuable content I read every morning.

My prediction for Quora is that it’s going to turn into a huge, important Internet property- it’ll break out of the Valley network easily and inevitably. The experience of Facebook going from college-to-college will inform a strategy of going topic-to-topic, profession-to-profession, and network-to-network. I can easily see how the Q&A mechanics would apply to many other things, especially the political blogosphere and Beltway insiders, the entertainment industry in LA, the media and advertising industry, as well as random everyday stuff. And of course, it’s one of the best executed products I’ve seen in a long time- the interaction design in the product is amazing.

Writing down all the things that product designers and entrepreneurs can learn from studying Quora would be many blog posts in itself. Like I said, I’m a fanboy :-)

Facebook app data shows stats for connected sites and products, including Quora
All the fanboys want to know: Quora’s been growing, but how fast? I recently realized that because Facebook sign-in is used so aggressively by Quora, they will end up getting listed as an app just like everyone else. As a result, Facebook (for better or worse) ends up publishing their DAU and MAU stats, which are then stored and graphed on services like AppData, AllFacebook’s stats service, and others. This establishes a lower-bound for all the core users who have authed to Facebook, but obviously doesn’t count users who bounce or who don’t sign up, etc.

I included the public listings for Quora excerpted from both AllFacebook and AppData. (Both are great services, I’d encourage you to try them). As you can see from the graphs, Quora is growing on a very nice clip, over 2X larger in MAU over the last month. Very nice!

Caveats: These numbers ought to be a lower bound since not everyone is going to either 1) register on Quora, nor 2) connect their Facebook accounts. As a result, I’m sure the real uniques number is much larger, but this is probably a good estimate of the active Quora community. I’ve also seen a lot of Quora answers in my search engine result pages, so I’m sure there’s easily a multiple that come purely to look at the answers and then bounce, that ought to be added to the totals. So again, think of it as a lower bound, but I imagine that the relative growth trajectory is right.

Again, this is really impressive growth and I look forward to seeing Quora’s progress continue.

PS. For the data geeks out there: If someone else does a more thorough analysis on their historic growth rate, sticky ratio, benchmarks/comparisons, etc., please write it in a comment and I’ll link you. And please point out if I’m misinterpreting these stats!

Just returning briefly from blog vacation to share a couple links and slides I had collected on retention metrics. There’s so little public information out there that I wanted to call out the various articles and presentations that actually do contain real data. Given the difficulty of getting exponentially viral on Facebook these days, most companies are focused on great lifetime value and making it work with big ad buys. Obviously good long-term retention is important for that.

After you calculate out some basic cohort retention analysis, where do you go from there? One key thing is comparing it to existing benchmarks to see if things are going well or badly. Below are some of the few public articles and slides with real data in them.

Here’s a collection of public retention data and discussion

First, from Daniel James of Three Rings:

Twitter retention analysis
RJMetrics, another analytics vendor, did this analysis of Twitter a while back: Twitter Data Analysis. Has some nice graphs like so:

Social gaming data for Facebook apps
And finally, Mixpanel based in San Francisco has some aggregated social gaming data:

• Social Game Developers Use Tutorials to Get Crucial Early Retention
• How to analyze Traffic funnels and retention in Facebook apps

Here’s a nice graph from them:

So what to make of all of this?

For now, you’ll have to look through this data yourself. I have a few rough notes I’ve written up about all of this retention data, and time allowing, I’ll publish some of them later. In the meantime, enjoy the presentations and links.

OK, back to blogging vacation :-)

Just a quick FYI on an upcoming conference – here’s the details if you’re interested:

Startup Lessons Learned is the first event designed to unite those interested in what it takes to succeed in building a lean startup. The goal for this event is to give practitioners and students of the lean startup methodology the opportunity to hear insights from leaders in embracing and deploying the core principles of the lean startup methodology. The day-long event will feature a mix of panels and talks focused on the key challenges and issues that technical and market-facing people at startups need to understand in order to succeed in building successful lean startups.

• Conference agenda
• Speaker list
• Where it is

I’ll be on a panel on Minimum Desirable Product with Dave McClure and others. We’ll be talking about the dynamics of incorporating design into a lean startup methodology, with all the difficulties and tradeoffs that entails.

25% discount if you use the link below:

Register for the conference.

I was recently invited to lead a session on customer acquisition and viral marketing at the First Round Capital CEO Summit (thanks Josh!). I wanted to share the notes I prepared for the discussion below – hopefully most of them will be self-explanatory.

I’m on blogging break right now, but I may expand the below notes into a series of posts when I have more time. Brb!

—
How to get have sustained viral growth:
– Have a great product (ideally in communication or social content)
– Convert user growth ideas into Excel-based hypotheses and clear user funnels
– Build and track each step of your funnels
– Get an initial stream of traffic (Adwords is OK)
– Optimize until every user is bringing in a new user
Timeline: weeks to months

Getting scientific about user acquisition:
– Start with your laundry list of acquisition ideas
– SEO, tell a friend, Twitter, etc.
– Convert into 2-3 testable hypotheses
– “Buy users for $1, monetize at $5”
– “20% of registered users will import addressbooks, >5 of their friends will register”

Viral loops in SaaS/enterprise
– What things do people share? What tools do they use for communication?
– files, wikis, Outlook, Excel, USB keys, etc.
– These are your viral channels (vs Newsfeed/Notifications on Facebook)
– If your value prop can align with a channel, then you might make it viral
– Case studies: Yousendit, Dropbox, Wikis, Basecamp, etc.

How quick-hit viral loops work for consumer products
– Cialdini’s “Influence: The Psychology of Persuasion”
– Quizzes: Social norms
– Top friends, eCards: Reciprocation
– 8 invites left: Scarcity
– But what’s the followup?
– Hide quoted text –

Value propositions for viral loops
– Best value prop is like Skype
– great for both parties (inviter and invitee)
– build deeply into the product (takes 2 to tango)
– Worst value prop is like lots of FB apps
– little to no value for the inviter/invitee
– lots of churn, feels spammy
– Sustainable viral growth is key for long-term value creation

Different acquisition models work for different kinds of businesses
– Vertical social networks -> SEO/SEM
– SaaS/enterprise -> SEO/SEM
– Consumer/communication/social content -> viral
– Themes, decorations for blogs/profiles -> widgets

Optimize your funnels by brainstorming levers
– Lets say you have funnel of Signup -> Download -> Install -> Fill out profile
– Lots of ways to improve
– change the order of steps
– remove steps
– combine steps
– use lightboxes, or longer pages, or progress bars, or lots of other UI tricks
– To optimize just the download-to-install step, you have dozens of options
– headline
– button placement
– “hero” photo or video
– target their OS
– size of download
– AIR
– small installer vs all-at-once
– installer filename
– etc.

Books and more resources
– Adam Penenberg, “Viral Loop”
– Robert Cialdini, “Influence, The Psychology of Persuasion”
– Tim Ash, “Landing Page Optimization”
– David King and Siqi Chen, “Metrics for Social Games” (Slideshare)
(lots of other resources on Slideshare)

Congrats to my sister Ada Chen, one of the first 10 employees at Mochi Media. I introduced her to Jameson (Mochi’s CEO and co-founder) back when she was first moving down to the Bay Area, and she turned down many other opportunities to go to Mochi. I remember she said she really loved the team, the opportunity, and thought she would learn a lot – which she has. I’m very happy it worked out for her. You can congratulate her at @adachen. (Oh and she’s also getting married this year to @sachinrekhi, another startup guy – congrats on that as well!)

From now, I know people will say, “omg you’re Ada Chen’s brother?” :-) It’ll be great.

When I first moved down to the Bay Area, I originally met Jameson, who was nice enough for me to crash at his old place in the Mission to attend the GDC. He ran Mochi off of a couple tables in his living room, where he lived with Bob in a work/live condo. They’ve gone a long way since then! Congrats to Jameson, Bob, and the rest of the team – well deserved.

Here’s some of the breaking news from the Wall Street Journal:

BEIJING—Chinese online game developer Shanda Games Ltd. agreed to acquire U.S. online game network Mochi Media in a deal valued at $80 million, furthering its global expansion ambitions.

Under the deal, which the companies expect to announce Tuesday, San Francisco-based privately-held Mochi will receive $60 million in cash and $20 million in shares of Shanda Games, a Nasdaq-listed, Shanghai-based company known for creating some of China’s most popular massive multiplayer online games.

…

Also more here at Techcrunch and Paidcontent.

Happy new year! I’ve been reading a ton of great books over the last month, and particularly the holiday break, and wanted to share them below with a couple comments.

Interaction design and rapid prototyping
Recently, I’ve been on a big kick to develop a much stronger opinion about design, particularly interaction design, and to build products prioritizing desirability over a business/metrics/optimization point of view. I’ve recently wrote about this perspective here.

Here are some of the books that have helped me in my thinking:

Inmates are Running the Asylum
This is probably my favorite book that I read all year. Alan Cooper‘s classic book that builds a business case on creating products from a user-centered view rather than business or technology. Introduces the definition of “interaction design” versus other design disciplines, the creation and use of personas, how engineers design software experiences, etc. Really needs to be updated for the agile programming movement, but still a very solid book.

IDEO’s Human-Centered Design Toolkit (PDF)
World-famous design firm IDEO published a toolkit documenting their human-centered design process. It’s longer than it could be because it lists all the methodologies inline, but it’s the deepest look inside IDEO’s design process that I’ve found. The important part is reading about how they go from user research to an insights framework to their “How Might We” questions that drive the creation of many low-fidelity prototypes. I’ve read a ton of books about personas but it wasn’t until I understood this process that I connected the dots on how to go from user research to prototypes to a final product – otherwise, it’s tempting for personas to become a useless artifact that doesn’t drive the product creation process. Read this, but my tip would be to skip through the methodologies on the first read – it’ll make more sense. Also, here’s a related PDF from the Stanford d.school here.

The Design of Business: Why Design Thinking is the Next Competitive Advantage
Artful Making: What Managers Need to Know About How Artists Work
Both of the above books cover similar ground, on how to relate innovation to the broader framework of ideating, designing, deploying, and growing successful products. In Artful Making, the discussion is around “artful” versus “industrial” processes, the former which emphasizes learning by doing and rapid prototyping, versus the factory floor process which emphasizes reliability and efficiency. The Design of Business looks at new product design as the process of moving from “mysteries” (new markets, new ideas) to “heuristics” to “algorithms” to “code” (efficiency-oriented, repeatable processes). The common idea from both books is that new product innovation is very different than metrics-focused efficiency processes, and shouldn’t be treated in the same way. That’s not to say you can’t have a strong, deterministic process around design innovation, but it just requires a different way of thinking.

Serious Play
This book deserves a much longer writeup, since I found it incredibly fascinating. Serious Play is about the notion that spreadsheets are to finance what mockups are to product, and what rehearsals are to theater. They are all models (or, if you prefer, prototypes) that allow people to simulate the future without incurring the full cost of actually doing it. The book touches on many of the first and second degrees of using spreadsheets, clay models, and other artifacts to drive decision-making, including politics, imperfections of models, and what kinds of industries excel at rapid prototyping versus others. Before reading this book, I never really saw the connection between spreadsheets and design mockups, but the author makes a compelling case linking the two as simulation tools.

About Face
Alan Cooper (see above) wrote a more tactical book about the actual “How To” around his Goal-Driven Design process, as mentioned in Inmates are Running the Asylum.

Just for fun
The below books are not necessarily related to startups, but I found them fun and compelling to read.

The Monk and the Riddle
Randy Komisar, a partner at Kleiner Perkins, wrote a philosophical book on life and startups a few years back that I would highly recommend. The core of the book is the idea that too many people try to live what he calls the “Deferred Life Plan,” where you do something you don’t love with the plan to eventually get to your real goals.

Coders at Work
Different profiles of engineers who have worked on important software projects.

The $12 Million Stuffed Shark: The Curious Economics of Contemporary Art
An economist dissects the world of contemporary art, the different players, what drives the economics, etc. I found this interesting from the perspective of art as a virtual good – his view of what causes high prices very much confirms this viewpoint.

Complications: A Surgeon’s Notes on an Imperfect Science
Atul Gawande provides a deeper perspective on what medicine is really like – the mistakes, the uncertainty – all the things you don’t really want to hear as a patient :-)

I also have an older book list here.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

Here’s a quickie roundup of the top posts from my blog over the last year, sorted by pageview. They are heavily skewed towards articles passed on to first time readers since most of my readership is via RSS.

A large number of them related to freemium, which tells you how much interest there was in making money in 2009 :-) Perhaps with the economy returning, there will be a shift of interest towards growth again.

Enjoy.

1. How to create a profitable Freemium startup (spreadsheet model included!)
2. Built to Fail: How companies like Google, IDEO, and 37signals build failure-tolerant systems for anything!
3. Free to Freemium: 5 lessons learned from YouSendIt.com
4. Product design debt versus Technical debt
5. Friends versus Followers: Twitter’s elegant design for grouping contacts
6. 5 warning signs: Does A/B testing lead to crappy products?
7. Freemium business model case study: AdultFriendFinder ARPU, churn, and conversion rates
8. Which startup’s collapse will end the Web 2.0 era?
9. 2009 conference schedule for the digital media industry
10. Does every startup need a Steve Jobs?
11. Why low-fidelity prototyping kicks butt for customer-driven design
12. What if interviews poorly predict job performance? What if dating poorly predicts marital happiness?
13. How to calculate cost-per-acquisition for startups relying on freemium, subscription, or virtual items biz models
14. 5 crucial stages in designing your viral loop
15. Age (and ARPPU) ain’t nothing but a number: Data on how age impacts social gaming monetization

To all my subscribers, thank you for reading!

Happy holidays everyone! I just wanted to make a brief return from a blogging vacation to show you a new Google search test where I’ve been randomly been assigned to the A/B test.

To summarize the main differences:

• Big blue buttons for everything
• Drill-down sidebar after a search
• Emphasis on filtering by time – so you can get the “latest”
• Search across their properties, including News, Blogs, Books, Forums, Shopping, etc.
• Features I haven’t seen (except in labs?) such as Timeline, Related Searches, Wonder wheel, etc.

Really a ton of changes!

Here are the photos: First, the homepage…

And here’s a search results page after an egosurf:

Here’s the expanded sidebar:

There are lots of changes, you can check out all the screencaps below:

UPDATE: Interesting – I’m noticing that the sidebar is switching between all text vs icons + large text, on a page-by-page basis. Plus they are changing the content around by quite a bit. Seems like they are still testing the exact nature of the sidebar.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

I had a recent quote in the NYT article on the Fitbit, and wanted to give a couple quick thoughts about using the device so far. (This isn’t a gadget blog so posts like this will be far and few between).

In general, I love the form factor and the fact I can clip it on and pretty much ignore it for the rest of the day. People that I show it to always remark on how small and cute it is, and I’ve gotten several tweets on how they’re jealous that I have on already. One funny thing is that I find myself checking it absent-mindedly the same why I check FML on my iPhone. In general, it has made me much more aware of my incredibly sedentary lifestyle, and the daily goal of 10,000 steps is a tough one to hit. I’m usually hovering around 5-6,000 steps at most, and have to actively work to get to 10,000.

The web integration is nice although I don’t find myself checking it that often. They just recently added some social features to it, and I’m sure when my friends, and family get these devices it will be fun to see how I am doing relative to them.

Anyway, it’s a basic device and does exactly what it’s supposed to do really well. I’m excited to see when it gets more gadgety and does more with the data.

For a more detailed review, check out the Engadget review or the CNet review.

What’s a minimum “test” of your product? And what are you testing?
A hypothesis-driven approach to product development dictates that you build as much as you need to test our your product, but not more and not less. But what are you “testing” your product for?

One possibility, as lean startups guru Eric Ries has stated, is to test your product for “viability.” He’s coined an important term, called Minimum Viable Product, and I’ll excerpt his excellent blog post below:

The idea of minimum viable product is useful because you can basically say: our vision is to build a product that solves this core problem for customers and we think that for the people who are early adopters for this kind of solution, they will be the most forgiving. And they will fill in their minds the features that aren’t quite there if we give them the core, tent-pole features that point the direction of where we’re trying to go.

So, the minimum viable product is that product which has just those features (and no more) that allows you to ship a product that resonates with early adopters; some of whom will pay you money or give you feedback.

He goes on to state that another example of this idea would be to set up a landing page and test for clickthrough rates and signup conversions, to see if there is any interest in the product. You could also stick a priced offer on the landing page to see how that affects peoples’ interest in registering for the site.

Viability is certainly one bar you can test for, but a related (and overlapping concept) is around testing product desirability. Let’s discuss this further.

Viable versus Desirable
In a previous post, I discussed an IDEO framework for how to think about desirability (user-focused) versus viability (business) and feasibility (engineering) – you can read that post here, called Does every startup need a Steve Jobs?

The idea here is that different companies often pursue products with different primary lenses – a business-driven company might try to assess viability upfront, thinking about metrics and revenue and market sizes. A feasibility (engineering) oriented organization might try to pick a super hard technology first (P2P! Mapreduce! Search!), then try to build a business around it. And a desirability-focused team might focus first and foremost on the target customer, their context and behavior, and build a product experience around that.

Thus, a Minimum Viable Product tends to center around the business perspective – what’s the minimum product I have to build in order to figure out whether or not I have a business? You might do that from testing signups on landing pages, try to sell products before they exist, etc. Putting up price points and collecting payment info is encouraged, because it helps assess the true viability of a product.

But what if you come from a human-centered perspective, and you want to build the Minimum Desirable Product? I think this is a subtle difference with big implications. A minimum desirable product (MDP) would focus primarily on whether or not you are providing an insanely great product experience and creating value for the end user.

Let’s define it as such:

Minimum Desirable Product is the simplest experience necessary to prove out a high-value, satisfying product experience for users

(independent of business viability)

To build an MDP, you will have to actually deliver the core of a product experience so that your customers can make a full assessment, rather than simply providing a landing page. Instead of measuring YOUR conversion rates and revenue generated, instead you might figure out the metrics of what benefits you are providing to the user. (I wrote about Benefit-Driven Metrics a while back) Similarly, you might make extensive use of qualitative research techniques such as the ones detailed by IDEO’s methodology card deck.

This also relates very much to Marc Andreessen’s definition of product/market fit, which he defines in purely market “pull” terms and not based on business ideas or viability. You could view the the Minimum Desirable Product as the simplest product that has a credible shot at providing that product/market fit.

Examples of MVP versus MDP
Let me make some quick distinctions about sites that might be Minimum Viable Products, but perhaps not Minimum Desirable Products, and vice versa.

• If you build a really viral social network that is profitable but has terrible user churn – you have built an MVP but not an MDP.
• If your profitable dating site gets lots of users to buy subscriptions at $20/month, but none of them find hot dates they were promised, you have built an MVP but not a MDP.
• If you build a magic box that spits out money whenever you hit a button, that is certainly desirable but not viable at all.
• If you create an amazing board game that your friends and family love and are addicted to, but you can’t get a game company to distribute it, you have created an MDP but not an MVP.
• If you have created a website with 20M+ uniques/month where people can tell each other what kind of sandwich they are eating, that has probably passed the desirability test but not the viability test.

(btw, I am writing this blog while drinking a soy latte at Cafe Epi in Palo Alto, but not eating a sandwich, for those who are curious)

Is desirability more important for consumer internet startups?
One of the key reasons why I began to think of this question is that it strikes me that consumer internet companies often don’t care much whether or not they have viable businesses in the short run. If you are building a large, viral, ad-support consumer internet property, you just want to go big! As soon as possible! This is particularly true for ad-supported sites where you need to break through a certain size to start talking to the brand ad agencies who can pay up on CPM. (More on that here) As a result of that, the goal becomes to hit product/market fit as soon as possible, and figure out the business model later.

Similarly, the key risk for consumer internet startups tends not to be technical risk or execution risk – it tends to be market risk. That risk may manifest itself as questions on whether or not there’s enough consumer value, or whether or not the market is big enough. These are things that may be proven purely based on desirability-oriented questions rather than getting into the business or technical side at all.

Minimum Feasible  Product?
The last though I will leave you with is, perhaps there are markets where the engineering portion is the most important – and thus the most important concept of Minimum Feasible Product.

For example, for a drug company curing cancer, the focus wouldn’t be on minimum viable product because if you have a cure for cancer, you’ll be viable. Similarly, you may not focus on desirability, because your product would clearly have pull from the market. You don’t need to do landing pages or user-centered research to figure out that curing cancer is a big deal from a business and user point of view.

Instead, the focus would be on Minimum Feasible Product – what is the smallest amount of work necessary to field a credible candidate for an “in lab” solution to the product?

For consumer internet, perhaps there are similar examples of this.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

Found this link and wanted to share it – thought it was an interesting argument. Quoted from Flurry, an iPhone analytics provider’s newsletter:

As all industry eyes look to the iPhone, the iPod Touch is quietly building a loyal base among the next generation of iPhone users, positioning Apple to corner the smartphone market not only today, but also tomorrow. In terms of Life Stage Marketing, the practice of appealing to different age-based segments, Apple is using the iPod Touch to build loyalty with pre-teens and teens, even before they have their own phones (think: McDonalds’ Happy Meal marketing strategy).

When today’s young iPod Touch users age by five years, they will already have iTunes accounts, saved personal contacts to their iPod Touch devices, purchased hundreds of apps and songs, and mastered the iPhone OS user interface. This translates into loyalty and switching costs, allowing Apple to seamlessly “graduate” young users from the iPod Touch to the iPhone.

An interesting thought, for sure.

Read more here.

David Shen, an Apple alum and now prolific angel investor, wrote me to chime in on my recent post on Steve Jobs.

UPDATE: I also heard from Kristee Rosendahl, who co-founded Apple’s Human Interface Group and worked directly on Hypercard, and posted her reply below as well.

I reposted David’s message below, with his permission, where he discusses the indirect effect that Steve Jobs has on the Apple design culture. He says Apple is still ruled by the business and engineering guys, but that his indirect effect is providing a central design vision as well as removing the politics around product design.

David writes:

great post, but i actually have a different viewpoint.

have you ever worked in a big org like apple? it’s filled with competing viewpoints, and is always run by business guys, never design guys. always design guys are relegated very far down the chain, and so thus engineering and business seem to drive the day on any decisions. this is often where we find “fake desirability”. [ed: fake desirability, which he defines “by this i mean that some people said they designed for users, but in actuality they only designed for themselves.”]

when i was at apple, it was certainly better than other companies. but still it was a guy who used to be at IBM germany who was CEO at the time from 1990-1993, after john sculley was removed. and design reported still a level or two below the CEO, but lucky for apple the culture itself supported UX and its products were consistently better.

when steve jobs came, he killed the political bullshit that made great products even better. everything runs through him and if he doesn’t like it, it’s too bad. so you have to suck it up to work at apple, doing steve’s bidding or else you will not survive in there.

he is a design dictator of the company. and it’s fortunate for apple and the world in general, that they have him because without his ironhand, the company would soon devolve back into a political, consensus driven company. it would still have great products from a certain point of view, but i doubt that they would ever have the game changing, superiority they exhibit now. committees would grow, politics would ensue, control battles would happen, and superior products would be hampered by all this. steve removes all that; he makes the final decision and pushes details that no one else would have the authority to push. and being at the top, you have to listen to him or else you’re fired. that’s it; end of story.

and thank god he is right most of the time.

so i would argue that benevolent dictatorships are the best form of govt in the world, including both for companies and for countries, where one person has both the right vision and the ironhand/cut-thru-the-bullshit attitude and style to do the right thing. think if obama ruled the US like steve jobs. he would just do the right thing, and nobody could do a single thing about it.

the probability of another steve jobs occurring is vanishingly small. i doubt that another startup could produce a steve jobs. it is a combination of intelligence, market savvy, strong personality, and ruthlessness that makes him successful. not many people can exhibit all those qualities to make it work.

believe me i have seen people try. but they just end up pissing everyone off and they fail when nobody can work for them, or they think they have supreme market savvy but really they are exhibiting “fake desirability”. remember that steve took decades to develop his ability to this day; a 20-30 year old is very very unlikely to have enough world experience to be able to match that. so maybe you could say that zuckerberg or larry/sergey are in that camp. but there are other tons of people out there who are not. so the probability of finding someone like that (or being someone like that) is pretty darn small….

Very interesting…

UPDATE: David added some additional thoughts in the comments.

I should clarify that I don’t think that Apple is run by business and engineering guys solely now. I think it’s probably one of the most balanced orgs, power-wise in any corporation I’ve seen. What other companies have their head of design reporting into their CEO? I can’t think of any!

But CTOs almost always report into the CEO, and certainly the business structures, like business units, general managers, etc. always do.

Authority and importance are often driven by how high in the org chart you are. Having a voice at the table as high up as possible means you get to be heard and your issues taken seriously, and your influence felt. It also means that the CEO has now told the company: “the guys who report to me are also the most important to me. That’s why they report to me.” If the design lead does not report to the CEO, then how can design truly have a voice in the strategic decisions of the company? It could only be translated through the voice of his manager, and so on, upwards until they get muddied and washed out by the time they reach the top…or just lost.

A clarifying point about indirect design influence: 

I actually think Steve has a direct influence on design on many products, and that the effect of this design influence creates what I’ll call “design philosophy inertia” which propagates through the org, across product lines and down product lines. This is where his indirect influence can be felt. But it is clear to me there are products that he cares most about, and these he will put his attention on all the time.

As I said in the post above, thank god we have Steve. I doubt we’d see the world be filled with such superior products without him.

UPDATE: Some thoughts from Kristee Rosendahl below on Apple and what startups can (and can’t) take away from process.

My comments about Apple have to be taken in the context of when I was there 1984-1990, as things continue to change there like any company. After 1990, I’ve been an external observer of Apple’s culture, just like the rest of us.

I think Steve is a design dictator when it comes to the products close to his heart. The good news is that his approach and sensibility is so baked into the culture of Apple that everyone inside Apple considers themselves design advocates. So other products get the advantage of that. It’s an amazing example of how leaders set tone, culture, and priorities. When I went to work for Apple, even as a consultant first, they gave me this little cubical with a Mac. Then the person said be prepared for Steve to walk in at any time and ask you what you are doing. The implication is that I better be able to defend my work at any moment. That set a tone from day one! He never showed up in my office, but talk about creating an environment based on that.

I also think Steve is in his own class, because he is not only a designer, he is an incredible marketer. I agree with you though that there is still lots of room to improve and elevate design within an organization. The issue will be that most CEOs can’t really talk about design. There are almost no classes in biz school that really address design – I sure hope that changes. So most biz or tech guys running the show are not apt to go there, its not their language, and not their safe zone. One of the major ways I have seen companies overcome this is with two partners as head – one who handles the biz side but totally appreciates and respects design, and the other is the creative lead who has respect and can partner with a business oriented person. The other option is to hire a really good design lead. Mostly, though, companies hire consultants, or agencies. When the job is done, there is no one in house to keep advocating from the top…design has got to be on the executive team and by the water cooler to make it work.

[…]

I would add that, in this discussion about design, remember Apple is a consumer products company. Most of what they are lauded for is their product design, ease of use, delight, coolness, etc. Designing real products people carry with them, work on, and use for entertainment purposes, is a far different design effort than creating a social media website. While both require design, their development time, designer’s skill sets and to-market time are not similar. Sometimes we need to make that distinction when we talk about design efforts in various different kinds of companies and start ups.

When a CEO who is starting up an online business says they want their product ” to be as simple as Apple”, we all know what that means. What start ups forget is how many people’s efforts and hours go into making Apple’s products that clean and simple. In my experience, it has been a real challenge to convey how much longer a simple solution takes over a complex one. A truly simple and elegant solution just demands more time and cycles than most people understand. So I’m delighted when you can hear designers talk about their process and the timeline. A simple product demands patience, lots of iterations and hence, additional expenditures.

At the same time, I’ve unfortunately seen small companies and many startups waste thousands of dollars and person hours spinning about the design of the product because they don’t have a clear idea of the core benefit. So in the end, they could have spent the same amount of money but had a very different outcome – a much better product. They need to get better at doing their homework… see attached Seth Godin post.

This is what Jobs understands and why removing the corporate bs is so important. The company politics or personal aesthetics can take down a good idea or product in no time, even in a tiny company. David also talks about “design by consensus” and I think that’s part of any startup. The group is typically so small, that to leave someone out of the design process early on doesn’t emphasize the “team” spirit of the start up. This can be a big mistake. Not everyone is involved in other parts of the processes – I don’t critique code, for instance. I leave that up to people who are experts at that function. But many people want or think that being part of the design decisions is part of their inherited right as an early team member – it’s fun, distracting and everyone has an opinion. My advice for a startup is to be very careful about how the process is handled. As a designer, this is part of my role as well – to design the process by which this can all happen smoothly. In the end you can get a mediocre design by consensus that looks cool to the internal team but does nothing for the potential customer.

A product’s design success also depends on whether you perceive design as merely a decorative skinning of the product once its developed or as an inherent part of the product development process. I get calls all the time from companies who are launching in 8 weeks, the product is in development, and they need a designer to come in to apply some look and feel to it. This is the antithesis of how Jobs works. And it shows. And it impacts the financial success of the product.

I think we designers also need to keep doing a better job at being part of the development teams. I’ve seen many a designer complain about having to attend development meetings – they just want wireframes and then they can do their magic. I think this is partially why developers have taken on some design roles. And I want to say here, I consider developers designers in their own right! Someone has to make choices early on, and if a designer isn’t there, the product gets developed either way. Designers need to get more agile, iterative, and more transparent in what they do. Today’s products demand that of us.

Lastly, I’m including my absolutely favorite post from Seth Godin. I think it sums up so well many points that would help both startups and existing businesses get a little shot of that Apple DNA. Seth’s observations are a good summary about how equally important fostering innovation is vs being an innovator. Steve Jobs does both pretty well. For now, pick one role and do it really well.

[Seth’s post on How to be a great client]

Thank you Kristee!

What does Steve Jobs really do for Apple?
I had a recent conversation on Apple’s incredible design culture and what it would take to create that in a startup. In many ways, it seems like an insurmountably difficult challenge to play the role of Steve Jobs, with his god-like sense of product aesthetics and interactions.

And yet, Apple has hundreds of products and experiences – hardware, software, HR materials, commercials, etc. Steve Jobs certainly doesn’t have time to work on the design of every Apple product, and of course has 35,000 employees to manage. So what does Steve Jobs really do, to create the amazing design culture at Apple?

And more importantly, can a startup hope to even start to capture the same kind of culture?

Well, let me give you my best guess :-)

IDEO’s product framework for Desirability, Feasibility, and Viability
First, let’s take a quick detour and talk about IDEO’s perspective on new product development – this is documented as part of their 100+ PDF on human centered design, but also recounted to me by a friend who works there.

The idea is that all products ultimately come from an epic struggle between three perspectives: Desirability, Feasibility, and Viability. IDEO focuses on new products from the desirability side, which means they think about how to make sexy products with clear value propositions, and think technology and business goals flow from that. Most of their Fortune 500 clients do not act this way, of course, which is why they have to hire IDEO.

Here’s the diagram included in their HCD toolkit:

The way this was retold to me is that these factors map into functional parts of a business:

• Viability = Business focus (marketing, finance)
• Feasibility = Engineering focus (technologies, agile process, etc.)
• Desirability = Design focus (customers, aesthetics, etc.)

Business-focused product perspective: Viability
For business-oriented products, the focus might be on any of the following:

• “hot markets”
• making money
• funding potential
• distribution
• metrics

The idea there is that you get to a product via one of these first-order items. A business-oriented entrepreneur might identify a market, then try to come up with a product within the market – for example, “wow, Zynga is making $250M/year, and fish games are big. I should come up with a social gaming product too.”

I would also argue that “corporate” thinking (including MBAs and biz plan competitions) fundamentally revolve around this approach – the most important thing becomes the analytical discussion around the business, rather than the core user experience itself. Financial metrics and market sizes become the dominating point of discussion – I would argue also that most venture capitalists fall into this bucket.

The big “religions” in this perspective are frameworks like Built to Last, Crossing the Chasm, Customer Development, Blue Ocean Strategy, even Efficient Market Hypothesis. You might also count Six Sigma, all the stuff in McKinsey quarterlies, etc.

Engineering-focused product perspective: Feasibility
For technology-oriented products, the focus might be on the following:

• programming language and development stack
• cool technologies or libraries
• engineering processes (agile or otherwise)

For people who use this as a first-order filter, you might end up with a line of thinking like, “BitTorrent is really cool, how do we build a business around it?”

I would also put engineering processes like agile into this, because that can easily become a first-order item in how to build a product as well. Agile won’t work for every team, for every product, in every situation, and yet it’s viewed as an all-purpose hammer – does that really make sense?

The big “religions” in this perspective are frameworks are agile, scrum, open source, etc. I might also count the “ecosystems” like Rails as a unique culture with its own set of beliefs and conventions. Frameworks like “Lean Startups” ultimately combine both Business and Engineering goals, via Customer Development plus Agile.

Design-focused product perspective: Desirability
For design-focused products, the focus might be on:

• context, culture, and goals
• customer goals and product experience
• design aesthetics and interactions

The first-order filter in this case might be “Sick people go to hospitals and have a terrible experience – how do we improve that?” The tools employed at this initial stage might include user research, development of personas and user goals, and rapid prototyping to explore many product concepts.

The big “religions” here are led by Apple and their aesthetics and standards. And of course folks like IDEO and their “design thinking” ideas.

How business and engineering goals encroach on the desirability of a product
Reading through the above, perhaps you have identified yourself as prioritizing one versus the other. And in general, the prioritization of the three different goals drives what kinds of product experiences you can build.

From the perspective of making a sexy, highly desirable product, you’ll find lots of objections from business or engineering:

• “spending money on visual design is too expensive”
• “polishing a product will make the process too slow”
• “this product is boring to implement”
• “can you redesign this product so we can build it in 1 week sprints?”
• “this target user is great, but we want the product to be more powerful and support more audiences”
• “but Zynga doesn’t do this, can you just copy them?”
• “why build so many prototypes that get thrown away? That’s costly and slow”
• “if you added X to this product, it would put us into strategic market Y”
• etc.

How do you handle questions like the above?

All of them are great questions, and of course the right answer means you have to find a balance in the approach. But what is the expense towards the core of your product experience?

Back to Steve Jobs – what does he really do?
Long story short, my hypothesis is that Steve Jobs is one of the rare CEOs who is very focused on product desirability. In battles with the business and technology goals, desirability will almost always win out.

So his role isn’t that of a designer, but rather Chief Design Advocate. This means:

• he makes it clear that products should be “insanely great”
• he recruits a top design team, and protects them from competing goals
• he is willing to spend money, adjust technology processes, all for the goal of highly desirable products
• he convinces financial analysts, industry pundits, etc. that product design is very important

To me, the amazing part about this is: Any company can do it.

Maybe not as good as Jobs, but they can decide to make it a priority – but few companies do. With the pressure of quarterly earnings, what competitors are doing, and employee aspirational desires, the focus moves off of killer experiences for customers – that’s no good.

If the above is true, then any of us can be the Steve Jobs of our team. Start by prioritizing design and desirability, and place it on a better footing relative to engineering and business goals. Learn the tools, develop your own religion, and start building great product experiences.

It almost sounds so easy!

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

I have been casually lurking on a new Google group focusing on the techniques around Lean Startups pioneered by Steve Blank and Eric Ries. Lots of fun conversations happening there.

Go here if you want to check out some of the threads going on. I’ve been following via the RSS feeds of new topics.

Here’s the info from the main site – thought I would give it a plug. The guy who runs it is named Rich Collins.

Let’s build a community focused on learning from lean startups.

I’ve recently been captivated by the ideas introduced by Steve Blank and Eric Ries. I’ve been reading their articles, watching their talks and listening to their podcasts. What I haven’t found is a community of other startup founders with a similar interest in lean startup strategies and tactics.

I would like to create a community centered around building lean startups. There will be a website with a forum, wiki, chat and Hacker News style social bookmarking. The focus will be on sharing battle stories and numbers from actual startup campaigns. I will also organize events for members to attend.

In true lean startup form, I created this website to gauge the interest in the formation of such a community. After 24 hours of having the site up, 131 people showed their interest by submitting their email address. As a result, I’ve created a Google Group where we can start the conversation.

Have fun! Hopefully there will be some interesting stuff to come out of it.

Amazon’s tabs are a classic example of product design debt and the refactoring process to pay it down

Incrementalism creates Technical Debt, and also Product Design Debt
Most startups these days build products using the various philosophies of agile – both in the formal sense but also the informal sayings of “deploy early and often,” “fail fast,” “ship and iterate,” etc. Coupled with A/B testing, customer development, and thinking through business problems in a scientific, hypothesis-driven way, you end up with a powerful cocktail of techniques to build a modern startup in the most iterative way possible. This kind of incrementalism is mostly great, and people should generally do more of it.

The interesting part is when you get a couple months into your product cycle. You often end up with lots of half-done experiments lying around, an infrastructure that isn’t built to scale, and a mishmash of code that needs to be refactored. Most engineers know that in this kind of a case, the best practice is NOT to rewrite your code, but rather refactor it continually and take down the so-called “Technical debt” so that it’s always under control.

However, there’s the other side of the coin, which is the product design. After you’ve added a ton of new features and stuck them all on the homepage, you create Product Design debt. The Amazon tabs at the top are a great example of this – you have a design philosophy built around tabs, you scale it as far as you can, and then you have to refactor your design.

Arguably, MySpace is a company that never paid down their product design debt, and their traffic has been impacted as a result.

Anyway, let’s dive into this topic more, starting with technical debt.

Technical debt
Most of my readers are probably familiar with the concept of technical debt, but just to re-summarize from this great article on the topic:

The first kind of technical debt is the kind that is incurred unintentionally. For example, a design approach just turns out to be error-prone or a junior programmer just writes bad code. This technical debt is the non-strategic result of doing a poor job. In some cases, this kind of debt can be incurred unknowingly, for example, your company might acquire a company that has accumulated significant technical debt that you don’t identify until after the acquisition. Sometimes, ironically, this debt can be created when a team stumbles in its efforts to rewrite a debt-laden platform and inadvertently creates more debt. We’ll call this general category of debt Type I.

The second kind of technical debt is the kind that is incurred intentionally. This commonly occurs when an organization makes a conscious decision to optimize for the present rather than for the future. “If we don’t get this release done on time, there won’t be a next release” is a common refrain—and often a compelling one. This leads to decisions like, “We don’t have time to reconcile these two databases, so we’ll write some glue code that keeps them synchronized for now and reconcile them after we ship.” Or “We have some code written by a contractor that doesn’t follow our coding standards; we’ll clean that up later.” Or “We didn’t have time to write all the unit tests for the code we wrote the last 2 months of the project. We’ll right those tests after the release.” (We’ll call this Type II.)

Of course, we are mostly interested in the second type. Eric Ries has a great article advocating for why it’s OK to Embrace Technical Debt. Another great article is from Joel on Software called Duct Tape Programmer. All of these articles are worth reading.

I won’t focus too much on the definition since those other posts do such a great job – instead, I think it’s worth talking about why an iterative approach tends to produce technical debt. I don’t think it happens all the time, but there’s always a temptation for it to happen.

Ultimately, the problem is that if you are trying to learn something about the business, and your technology is meant just to support that experiment, 99% of the time it’s not worth it to do things the “right way.” The reason is that you don’t know if something is going to work, and as a result, you don’t want to invest in scale or perturbing your entire codebase for something that might be disposable. So instead, you just put a 10% or 25% version of the product out there (now commonly referred to as the Minimum Viable Product) and do as little coding as possible to get there.

The problem is, when the feature is successful, very rarely is a team going to then go back and rewrite it – every experiment creates more questions, and the temptation is to move on to the next question.

Product design debt
A similar problem to this is Product Design debt, which impacts the user experience rather than the underlying technology. The same temptations that lead to technical debt also lead to product design debt, because it’s always harder to do things the “right way” and it’s almost never a rational investment of resources. Show me a site that has great visual appeal, and I’ll guarantee that they don’t A/B test.

Product design debt happens because of scenarios like the following:

• “I want to test this new feature, where should we put it? How about the tabs?”
• “Can we throw this experiment on the homepage and see if people click on it?”
• “Our navigation is kind of getting out of control, but if we fix it, most of the site’s features will lose a ton of traffic”
• “We just added a Lists feature and we want to promote it, can we just add a button next to everyone’s name?”
• “Yahoo just bought our startup and they are going to stick us on their homepage!”

(just kidding on the last one)

The point is, as a product experience grows deeper, at some point the initial design philosophy of just adding more links to a page or more tabs or more buttons just stops scaling. Yet it’s often hard to reorganize the whole site, especially if it means taking a short-term dip on traffic, so the “safe” thing to do becomes to incrementally add things until the user experience is horrible.

Kudos to Facebook for looking at their product and deciding that they needed to refactor everything first into a big newsfeed stream of “stuff,” and then all their features into a generic container of apps. They’ve also done a lot to actually remove options from the menu and navigation.

Why homepages becomes a Las Vegas visual experience
Incrementally-developed UIs that are never refactored often turn into a Las Vegas visual experience over time. Ya know, something that looks like this:

Why does Vegas look this way? I’d speculate that all these buildings are ultimately infringing on the public good of aesthetics, and light pollution becomes a tragedy of the commons. If all of those buildings were to power down, it may be that the relative distribution of business would remain the same, but we’ll never know since that will never happen :-)

Important navigational areas like homepages, inboxes, notifications, etc are all the same way. Each incremental menu item is not a big deal, and provides a lot of value downstream, but a slight incremental cost. But do this enough times, and you’ll start to pollute the overall design aesthetic, which is a public good that all features share.

For startups, this shouldn’t be a huge problem because you should have a product person who manages the whole experience and can resolve the public good problem. But there’s a danger in bottoms-up startup cultures where anyone can throw up an A/B experiment, which on one hand is great, but on the other hand creates UX pollution. The other class of cultures where this becomes a problem is short-term optimizing cultures, which may have a “feature of the week!” they want to focus on, which they need to exaggerate each feature each week.

For established companies with multiple teams competing with each other, this may become a key problem because then it really is a public good within the company.

Product types that are most susceptible to design debt
Ultimately, I think product experiences that provide a million little features are the ones that need to watch out the most.

This means:

• Social networking and Community sites that want to unify chat, forums, polls, videos, blogs, etc.
• Portals that want to unify news, communication, tools, etc.
• Games that want to unify lots of different missions, communication, characters, revenue-generating activities
• Retail products that want to unify lots of product categories and SKUs
• Classifieds sites that want to sell lots of different services, products, people, etc

All of the above products are hard to design for because they are meant to be open and support lots of diverse activities, but refactoring the UI constantly becomes a strong need as the initial navigation paradigms probably will not scale.

I wrote an article a while back specifically on social community sites, called Social Design Explosion.

Ideas for when and how to pay down product design debt?
For entrepreneurs out there who are building metrics-driven products but also committed to a great user experience, I would love to hear when and how you pay down the product design debt. Please comment!

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

Upfront design and agile don’t mix well
It’s an interesting problem to try and mix traditional design tasks – visual polish, user research testing, etc. – to an agile development process. A weekly development cycle doesn’t leave much room for several iterations of mockups, the immense effort of recruiting and interviewing users, and all these other important tasks.

Anyway, I was sent this recent link I’d encourage you to read on 12 emerging best practices for adding UX work to Agile development.

Here are the list of 12:

1. Drive: UX practitioners are part of the customer or product owner team
2. Research, model, and design up front – but only just enough
3. Chunk your design work
4. Use parallel track development to work ahead, and follow behind
5. Buy design time with complex engineering stories
6. Cultivate a user validation group for use for continuous user validation
7. Schedule continuous user research in a separate track from development
8. Leverage user time for multiple activities
9. Use RITE to iterate UI before development
10. Prototype in low fidelity
11. Treat prototype as specification
12. Become a design facilitator

In general, the best practices are about taking the down the level of fidelity in the design process and trying to work ahead of the engineers so that they get the fast feedback they need. Definitely worth reading.

Seattle is a great tech city
Since I was 5 years old until 4 years after college, I called Seattle my home, and technology was intertwined with my childhood. As a kid, I found lots of avenues to my formative years in computing, including access to gopher and telnet via Seattle Community Network, the pre-web BBS scene, and a 5th grade classroom filled with Macs. As a college student, I got to work at various tech startups and ended up at a VC firm after I graduated. There’s not a lot of cities that have the ecosystem to have given me opportunities like that – maybe half a dozen at the most, and Seattle is certainly high up on the list.

Ultimately though, I left after 2006 – it took a lot of soul searching but ultimately one question got me over the edge. Let me explain what that was.

The question that got me to leave Seattle
As I pondered staying or leaving Seattle, I did a lot of thinking about the city from a startup context and what was working and not. Obviously it’s great to have companies like Microsoft, Amazon, Real, and others there – it produced a wonderful tech ecosystem that is thriving and growing every day.

But in late-2006, the social networking world had caught fire, and I wondered:

Post-bubble, when was the last time Seattle produced a world-changing consumer internet company?

And try as I might, I couldn’t shake the idea that while the rest of the tech world in California was producing YouTube, MySpace, Facebook, Google, and others, Seattle had Amazon and sort of stopped.

I wasn’t sure that I would be able to answer WHY, but I packed my bags and figured I’d figure out a theory at some point. A few years later, thinking about the question now, I think it has a lot to do with the kinds of companies being built in Seattle.

Different kinds of companies – Commerce versus Community
My current hypothesis is that Seattle has a strong history in retail and commerce, which has influenced the kinds of companies that are started there. Obviously you have Amazon, but you also have Eddie Bauer, Blue Nile, Nordstrom, Costco, Starbucks, and numerous other online/offline retail businesses there. There are also lots of transaction-focused startups based in real estate (like Redfin) or travel (Expedia).

These retail and transactionally-focused businesses are great money-makers, but because they target in-market buyers for a particular good or service, it means that you’re not really building a huge audience. You end up with the <10% of the general population that is in-market for buying a diamond or plane tickets or a house, not a viral and sticky UGC site you visit every day.

The classic way to build a huge audience is to focus on ad-driven businesses in the world of communication or content publishing, and there just aren’t that many of them in Seattle. (Though congrats to the Ben Huh for marching his horde of cats in this direction – the Cheezburger sites have the #1 traffic slot in Seattle right now) If you look at categories like social networking or YouTube or Twitter, these are more like everyday tools that hundreds of millions of people might use every day to communicate or find the content they want. Those are mass audience driven businesses and end up being high-variance outcomes – you end up with huge hits and also big failures because you need more money-losing years to build up the audience necessary to monetize at the rates you want. (just look at Imeem’s recent firesale even as they had amassed tens of millions of active users)

Different types of expertise – SEO versus viral/social
Similarly, the above influence also drives the skillset involved for one of the key startup goals: Driving traffic. My working hypothesis for Seattle is that it’s a very strong SEO-oriented community, and you have many of the top experts living and working there. The reason, of course, is that retail and transactional sites are mostly found via Google, and it makes sense to develop a skillset around getting that traffic for free rather than paying the search engine for it.

That’s great, but that also closes the door for the all-important knowledge of the viral loop that companies in social gaming are learning now, and what social networks companies learned before them.

For that reason, much of the social gaming and social network action happens down in the Bay Area.

Comments?
In short, years later I think I’ve mostly answered my own question – my hypothesis is that Seattle hasn’t produced mass audience consumer products mainly because it’s focused on down-to-earth charge-users-for-a-product types of businesses that are more transactional than community. I don’t think that’s a good or bad thing – just as you’ll get more biotech in Boston, there’s a specialization in Seattle around commerce/retail. But if you’re doing a social UGC thing, the Bay area is the best place to be.

Seattle folks (or otherwise): Do you agree or disagree with the above? Let me know in the comments – would enjoy hearing your thoughts.

UPDATE: For all the people who think I’m being a Seattle-hater, here’s a similar analysis for the Bay Area: Does Silicon Valley noise detract from long term value creation? It’s a related piece and discusses some of what I’ve noted since being down in SF.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

I’ve been blogging less and less
As many of my readers may have noticed, I’ve been blogging less and less lately – it used to be multiple times a week, then it became once a week, and recently I’ve been blogging once every other week or so. I’m sure I can keep up that pace for quite a while, but it certainly makes for a less interesting blog :-)

Anyway, some of the reasons why I’ve slowed down in my blogging:

Blogging is more fun when you’re meeting people from lots of diverse companies and industries
When I was doing my Entrepreneur-in-Residence gig, I had an excuse to do lots and lots of meetings with people from across the digital media industry. On a single day, I might talk to companies in mobile, ad infrastructure, payments, social networking, games, and more. That was a great opportunity to blog because it’s easy to see connections and talk about ideas across industries. I don’t do this much anymore, so it’s harder to come up with these observations.

Getting deeper and narrower results in boring blog posts
Getting deeper and deeper in an area is a key part of the startup experience – you learn lots of weird things about your particular project, your particular target audience, and your specific industry. This doesn’t translate to great blog posts though, because most of what you learn there is completely inapplicable to other peoples’ situations. Instead, you get articles that are too “inside baseball” and esoteric.

Long blog posts are hard (and get harder over time)
Sometimes I really appreciate Twitter’s 140 character limit because it forces you to be short and sweet. A blog post, in particular my blogs, go the other direction. Over time, this has become a big pain in the ass since I’m not as comfortable posting one or two paragraph blog posts and instead go overboard with essays. I should probably just come up with a word limit and try to keep things down to a more reasonable size instead :-)

News-driven versus writing whatever
Another is getting inspired to write something – it’s a lot easier to write to comment on something in the news, versus just thinking about a particular topic and writing something great there. It’s always helpful to have some inspiration.

Potential changes?
From the above, it seems like a couple experiments might make sense. A big thing I should do is probably to write shorter things, and maybe do more news commentary. We’ll see if that helps at all :-)

Anyway, less excuses – back to blogging!

Followup to Ning’s Viral Loop article

I was recently sent a copy of Viral Loop by Adam Penenberg, which just came out. I was first introduced to Adam in early 2008, when Marc Andreessen wrote us both while Adam was starting to write an article about Ning and their viral loops. That article was ultimately published in April 2008 as Ning’s Infinite Ambition, which you should read if you haven’t. After the article, Adam subsequently spent more time researching the topic, ultimately resulting in the book. I finished it and wanted to share a high-level summary and also talk through some points that the book brings up.

Summary
The book mostly covers a series of case studies from both offline and online companies. These include detailed dissections of viral companies from all stripes, including:

• Offline: Tupperware, Ponzi schemes
• Andreessen’s companies: Mosaic/Netscape, and Ning
• Bubble era companies: Hotmail, eBay, PayPal, HotOrNot
• Web 2.0 startups: Flickr, YouTube, MySpace, Bebo, Tribe, Tagged
• Widgets and apps, etc: Facebook, Slide, RockYou, Zynga

Some of the companies get pages and pages, and others just get a paragraph or two. But there’s a lot of stories that were new even to me, which is always a good sign, since I tend to love reading this kind of stuff.

The book also covers a bunch of high-level concepts about virality, such as the viral coefficient, viral loops, RockYou’s model for calculating virality, etc. All in all, a useful intro to all the major concepts in the field. It’s a great walkthrough of the history of viral companies since the late 90s when some of the formalizations started to happen.

Metrics-focused virality versus not?
One of the interesting distinctions that isn’t made in the book is the trend of startups who use quantitative techniques to optimize their virality versus products that went viral through other means. In particular, a lot of modern techniques are borrowed from the world of leadgen, ecommerce, and advertising, including:

• Formally defining landing pages (and using associated techniques)
• Creation and formal creation of funnels
• A/B testing
• Extensive use of analytics and targeting
• Deep understanding of email marketing, deliverability, and addressbook importing

From my personal experience, it seems like a lot of these ideas about virality ultimately originated from a few small teams here in the Bay Area who have now helped generations of viral companies succeed.

To me, the most important work in metrics-based viral marketing came from these companies below – I’ve listed the companies along with “descendent” startups who took the culture, playbook, and to build the next group of viral companies

• PayPal (Peter Thiel and Max Levchin)
  • PayPal mafia (Slide, Yelp, YouTube, Linkedin, Geni, etc.)
• Jumpstart (Greg Tseng and Johann Schleier-Smith)
  • Crushlink, Tagged, Hi5, others
• Plaxo (Sean Parker)
• Tickle (James Currier)
  • Ooga Labs (Medpedia, Wonderhill)
• BirthdayAlarm (Michael Birch)
  • Bebo, Flixster

There is lots and lots of overlap amongst this group above, and people cross-advise each others’ companies. Let me also caveat that the list isn’t exhaustive, and there are plenty of important VCs, advisors, and entrepreneurs that “get it” and help cross-pollinate between companies. In particular, I’ve found that the PayPal folks are involved in a tremendous number of companies in the Bay Area, and have been teaching their various companies to go viral for quite a while.

That said, I believe that the social relationships above have become less important over time to startups, as the knowledge around designing and optimizing viral loops has become more widespread. Certainly the Facebook economy has taught a wider generation of 20-something developers on how to build highly viral applications, with or without the help of the folks above. I’d note that some of them aren’t as numbers-oriented as copycat-oriented, but it’s still working for many people. As a result, I think the Bay Area is set up nicely to create the next generation of web companies as the bench in this area has gotten very deep indeed.

Who am I missing? Email me or let me know if I am in the comments. Or if someone on the above list would like to graciously identify who taught them the viral playbook, I can help trace the history further :-)

“Viral Loop” stays high-level
One aspect, both good and bad, about the Viral Loop book is that it stays pretty high level. As mentioned above, even after you understand what a viral loop is, you have to understand the tools of the trade well enough to go execute one. Learning the ins-and-outs of direct marketing takes a long time, especially to become an expert.

Adam does a great job keeping the book high-level and relevant to people both inside and outside of the industry, but certainly it doesn’t go into any of the details that have to be mastered to do the actual execution part.

It is for this reason that the total supply of viral experts will always be relatively constrained. Anyone worth their salt would likely be working on an amazing project, early on in the team, rather than working for an established startup. Instead, what tends to happen is that the community operates on a “money + knowledge” type of relationship, in which successful viral experts advise new startups to provide both angel investment and advisory.

The limitations of viral loops as a force multiplier
Another thing that isn’t discussed much in the book, which I think is very important, is the limitations of viral loops. The quantitatively marketed companies that I mention above certainly have their successes, but similarly, many of them are plateauing and failing as well.

The reason is that there are some important factors that are not well-understood by the extended community.

First, I refer to this great presentation by Siqi Chen (of Serious Business) and David King (of Green Patch), called Metrics for Social Games:

The first slide contains a deep truth: Metrics are a force multiplier. If you don’t have a great product, then you won’t get anywhere. But if you have a great product, then it help you build a huge business.

I’ve written about a similar concept in a blog post called Creating value versus optimizing revenue.

Hitting saturation in viral networks
Another important limitation is that there’s a finite number of users out there, and after you churn through all of them, all you have to look forward to is the long plateau. I first wrote about this in my post Facebook Viral Marketing: When and why do apps “jump the shark.”

I wrote that post back in March 2008, and a lot has happened on the Facebook platform since then. This includes an incredible growth rate of the underlying platform itself (now hitting 325 million monthly actives), the appearance of Social Gaming, and it turns out that the current model to beat on Facebook comes from Zynga. They get around the jumping the shark issue by releasing lots and lots of games – 17 on Facebook, 9 on MySpace, 8 on other networks, and 5 on iPhone. And more to come every day :-)

Although it doesn’t seem like much of a problem for most companies to hit the saturation ceiling in the networks they are operating in, it is a huge problem for VC-backed startups because then the story stops being about growth. So for the entrepreneurs who are working on their startups, it becomes important to hit a product/market fit early, and scale then, rather than prematurely going viral without a long-term product direction.

Buy the book here
Hope you enjoyed the post, and you can buy Adam Penenberg’s Viral Loop here.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

omg she’s getting scammed by a duck!

Techcrunch on social gaming scams
As everyone knows, Techcrunch recently published a provocative article called Scamville: The Social Gaming Ecosystem of Hell. Most people will have already read this article, but just to summarize, Arrington argues the following:

• Social gaming companies (particularly Zynga) are making their revenues in a “completely unethical” way
• Users are getting scammed by the offers
• There’s harmful cycle where the scammiest companies earn more revenue, then buy more ads, then scam more people
• Similarly, some users opt in to offers and then cancel, lowering their value, driving out advertisers
• And finally, the industry is in total denial about this

It’s a compelling article, and I would encourage everyone to read it. There’s also another followup article on publishers who decided not to go the offers route, HotOrNot and PlentyOfFish.

Let’s dig in
I am very sympathetic to Arrington’s views, and investigated the issue over a year ago – here’s my blog post from August 2008 on the topic: Super Rewards and the leadgen side of Facebook virtual currency – can it last?

The more I dug into the issue, the more nuanced I decided it really was – things weren’t all bad, actually. In fact, I’ve come to believe that there are plenty of advertisers where this is working for them, plenty of consumers who are happy as well, though these offer guys are leaving a trail of unhappy users.

It’s clear that of all the issues, the user experience must be fixed. And after the user experience is fixed, I think we’ll still be left with a thriving industry, though people may be making less money than they are right now.

I want to drill into some of Techcrunch’s assertions and go one level deeper to look at the evidence.

It does makes the user experience suck
First off, I think everyone is clear that the way offers run right now, they are very confusing for users. If you search for “zynga sucks” on the Facebook.com domain, you get lots of angry complaints, most them about offers. In fact, I did an article a long time ago and got a bunch of random angry comments that clearly had just been searching for SuperRewards, completely unsolicited.

This was a long time ago, so I hope their service has changed a lot – but here’s a sample:

The post is in regards to Super Rewards. From a game user standpoint, I did the offers more when Super Rewards was not managing the offers. Super Rewards is slow to respond to problems from users, and requires proof of the completion of the offer in ways that the offer itself does not require you to do. For instance, to receive points a mini-game was played, many many offers were reviewed, then the game results were given. The offer states that the points would be awarded once the user reached the results page. If the points are not received, you are supposed to file a request for review. Well, Super Rewards would not take as proof of completion all the information from the results page. They even argued with what the results page displayed, even though it was cut and pasted directly from the site complete with the web address. Instead, they wanted two emails, one confirmation email and one confirmation of the confirmation email EVEN THOUGH DOING EMAILS WAS NOT REQUIRED BY THE OFFER.

Here’s more:

I got stung by them 10 days ago. 440 points for a Discover Card application. I applied, and I am holding the card in my hand RIGHT NOW. They say Discover has no record of recieving my info. Really? Well why did they send me a card then? A$holes.

As one of the ripped off customers of Super (assholes) Rewards on Facebook, I have to say that thier service is a total and utter crap to say the least. Of the offers I have spent time filliing in I have only recieved points for 2 out of the 20 or so offers that I have completed.
Any complaints either get a automated response or no response at all.
There are now groups being formed on Facebook complaining about this type of action. I hope the group action gets up and going, these crooks need to be shut down.

Clearly this is not what SuperRewards wants, nor their game publishers, nor Facebook. And like I said, I hope SuperRewards has cleaned up their service since then.

The folks over at Gambit have written a solid article addressing these issues head on, where they discuss 3 game ending user complaints:

1. “I did your offer but didn’t get my points.”
2. “I completed this offer even though it took forever and now I’m getting spammed.”
3. “I completed this free offer and now I’m being charged all this money.”

The article goes on to discuss why resolving these issues is an important part of the game developers job, and how they can’t just say “oh that’s monetization” and not care about it. These user feelings ultimately come back into the game, and create problems long-term.

I would like to see more of the offers companies directly discussing and addressing the user experience problem openly – I think that will ultimately be the positive result of all of this dialog.

Everyone should be in agreement that the offers experience sucks, but no one is willing to do much because making these changes would mean a short-term monetization hit. It’s a Prisoner’s Dilemma where as long as the big offers providers continue in their ways, everyone wants to match them for competitive advantage. Thus my argument that the only player that’s able to get everyone in line would be Facebook.

It does seem to be working for advertisers
Arrington also makes the argument that the offers industry isn’t working for advertisers, and will eventually cause the monetization to crater. After talking to a lot of people on the issue, I just don’t know that it’s true, to my surprise.

Here’s the quote from the Techcrunch article:

And some users aren’t dumb, either. For every user who gets tricked into some fake mobile subscription, there’s another who can beat the system. That’s where the legitimate advertisers, like Netflix and Blockbuster, get hit. Users sign up for a free trial with a credit card, get their game currency, then cancel the membership and start over.

I specifically asked Jay Weintraub to look into this problem earlier in the year, and was genuinely surprised by the results. I figured that it was all a house of cards, but Jay came back to me with the idea that in fact it’s probably working (at least somewhat).

This is definitely required reading for anyone thinking about these issues. Jay did a great job breaking down the issues.

To summarize his analysis:

• The offers ecosystem on Facebook shares some surface similarities with the “Free iPods” incentivized offers industry that ultimately imploded (just read about Adteractive, Gratis, and similar companies for background)
• The volume of leads being produced by Facebook apps is so large that it’s unlikely that the crappy performance is just being hidden in the volume
• However, the pricing on Facebook will likely go down, and companies will make less money in the long run
• The offers may actually be performing, with the working hypothesis being that users actually choose the offer to fill out, versus the “Free iPods” case where they are run through a forced set of offers
• Also, long-term gameplay encourages accountability and repeat purchase

I think all the above points are surprising, and probably right.

Advertisers may reprice, rather than leaving Facebook
Arrington’s also argues that the bad leads will ultimately drive out all the advertisers. He writes:

Netflix has a policy of only paying for a user once. But game developers use a complex set of partner chains to launder these leads and try to get them through for payment. Netflix sees an overall lowering of quality and pays less for leads. Game developers, desperate to monetize, then search for ever more questionable offers to make up the difference. In the end, the decent advertisers are out, and only the worst of the worst remain.

My question is, why they won’t simply be repriced?

If an advertiser is buying leads at $3, but half the users cancel their orders, then why not just reprice down to $1.50? In fact, the best advertisers probably have the best products, and you might argue that their danger of cancelation is actually less than companies selling niche crappy stuff. Similarly, the Facebook leadgen infrastructure is now a big enough animal that advertisers may want to participate just to drive up volume. Even if an advertiser ends up with an additional $10M with no margin, they might do it anyway just to get more heft into their business.

So I agree with Jay’s argument that in the long-run, these leads just all get repriced, and the same set of advertisers (plus or minus) will remain.

Part of my positivity here is my direct experience buying Facebook advertising, which has actually been high-quality and high-conversion, for the most part. I think that the fundamental traffic is good, and thus the offer advertisers can see the same results, if they aren’t obnoxious about it.

It does create value, through product bundling
The other question is whether or not there is actually any underlying value to offers. And as I wrote in a post yesterday, offers theoretically should be good for everyone, the same way that Amazon and Netflix recommendations are good for consumers. The problem has been the execution, due to user experience issues.

Arrington seems to think, however, that getting users to pay more for the offer to subsidize the virtual good is a bad thing. He writes:

[…] Most of these offers are bad for consumers because it confusingly gets them to pay far more for in-game currency than if they just paid cash (there are notable exceptions, but the scammy stuff tends to crowd out the legitimate offers). And it’s also bad for legitimate advertisers.

I think the above statement doesn’t correctly describe how and why offers can add value overall. I won’t repeat the entire post here, except to give the outline of the argument:

• Amazon recommendations is good, and product bundling as a whole is good
• How do you define a “good bundle” versus bad? How will we target offers in the future?
• How does offer + virtual goods bundling actually work?
• Only 1% of people buy at an ecommerce site

If you haven’t read the article, check it out here.

What will happen next?
My working hypothesis is that the following things will happen – and it might take less than a year:

• The offers industry will continue to grow, the # of players will continue multiplying
• This will mean that the competition for doing leads will be cut throat, and no one will think long-term
• Ultimately, Facebook will intervene to preserve the user experience and make users feel safe in the checkout line
• If they decide not to do it themselves, they will heavily regulate the situation
• Otherwise, they will just make their own “clean” version, potentially by building out the Facebook ads into having landing pages, transaction forms, and redirects, rather than just sending clicks

Either way, I predict it will not end well for most of the leadgen players, unless they clean up fast.

If Facebook regulates, I would like to see them do something like this. Think of it as the FDA food packaging guidelines, but instead of calories it’ll talk about total cost to the consumer.

More reading
Here are some of the related articles that I would recommend anyone interested – they are from the view of the monetization gurus, and looking at advertiser-related performance, rather than user experience.

• Will social payment platforms really work long term? (guest by Jay Weintraub, leadgen expert)
• F*** your game offers (by Noah and friends at Gambit)
• Incentive Promotion 2.0 (also by Jay Weintraub)
• The impending doom of Facebook apps (by Niki Scevak, Jupiter Research)
• The economics of app monetization (with Noah Kagan, Gambit)

and two recent posts I just did related to the same Techcrunch article:

• How social gaming offers create value for everyone
• How Facebook could clean up the offers industry

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

If Facebook doesn’t clean up the offers industry, then this guy will

As a quick follow-on of my last post on How social gaming offers create value for everyone, it strikes me that what the industry needs to survive for the long-term is for one of the big players to break out of the stalemate of zero information sharing, and start advocating for sustainability.

Why all the advertising and leadgen companies hide their information
One of the big problems for the advertising and leadgen industries is the massive lack of information sharing between different parties. The reason is that ultimately, there are really just two parties involved:

• The paying customer
• The company providing the end product or service

But then lots and lots of middlemen get involved, including:

• Agencies / SEMs
• Ad networks
• Publishers
• Infrastructure providers
• Data providers
• etc.

Everyone in that extended chain are just middlemen, and their job is that for every $1 of profit, they want to outmaneuver everyone else in the stack to get as much of that dollar as possible. So if an ad campaign is doing really well, the agency doesn’t want to tell the ad network, for fear that the ad network will raise their rates. On the other hand, the ad network can’t figure out which of the publishers in their ad network actually deliver good performance.

This all sucks, and requires a central party to think long-term. That player might ultimately just be Facebook, but could be a publisher like Zynga (though I doubt it).

What information could be worth exposing
In general, I believe the key to thinking long-term on the offers industry would be to expose all sorts of feedback information, out in the open, at a granular level.

Users would also be able to get information like:

• What are they actually signing up for?
• A standardized view of every offer, like a checklist, similar to FDA mandated food packaging guidelines:
  • What is the 12-month cost of this offer?
  • What is the $ value of this offer to the advertiser?
  • Is this a subscription, yes or no?
  • Am I going to get emails?
  • Am I going to get a phone call?
  • Is my information getting shared with any other parties?
  • How can I cancel? (and this should be standardized too)
  • How do other users feel about this offer?
  • What is the cancelation rate?
  • How do I get customer support if I opt in to this offer
• Every offer should link to an “advertiser profile” on Facebook, with comments, ratings, etc.
• Facebook should be able to instantly ban specific advertisers and offers from ever coming up across all of Facebook

For advertisers and everyone else, they would get to see information like:

• Where are my offers showing up? (by app)
• What kinds of users are filling out my leads? (demographics, geo, etc.)
• What is the $ incentive for users? (by app, by $ amount)

Similarly, there is soft information like:

• How are users rating the app?
• How do they feel about the particular offer?
• How often engaged are users? How much churn is in the app?
• How often do they repurchase virtual currency?

For all of the above, I think a lot of companies would hate it in the short run, and a lot of dollars might be banned, but long-term, this would be better for the overall ecosystem.

Let’s hope that something like this happens!

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

The happy meal is the quintessential version of great product bundling

How offers add value
There have been a lot of conversations about the evils of offers in social gaming, and one thing that’s getting lost in the conversation is the potential for offers to actually generate value overall.

Ultimately, offers are about “product bundling” and it adds value to the economy the same way that any product bundling adds value – by giving people more of what they want, often for less. And naturally, some configurations of different bundles are more effective than others, as we’ll see below.

This post will touch on a couple topics:

• Amazon and “relevant” bundling
• How to define good product bundles
• What’s actually happening with offers and bundling
• Solving the 1% ecommerce problem at the Point of Sale

Let’s get started:

Amazon.com and product bundling
When you are shopping at Amazon.com, and you’re in the process of buying a book, and different book is recommended, how do you feel about that? And even more, if you happen to decide you like both books and want to buy them, and Amazon is willing to give you an aggregate discount, how do you feel?

I think that intuitively, the cross-sell and bundling that happens on Amazon is great for the customer experience, and exemplifies the good side of product bundling.

Here’s some additional information about it from Wikipedia:

Product bundling is a marketing strategy that involves offering several products for sale as one combined product. This strategy is very common in the software business (for example: bundle a word processor, a spreadsheet, and a database into a single office suite), in the cable television industry (for example, basic cable in the United States generally offers many channels at one price), and in the fast food industry in which multiple items are combined into a complete meal. A bundle of products is sometimes referred to as a package deal or a compilation or an anthology.

The article goes on to say that the strategy is most successful when:

• there are economies of scale in production,
• there are economies of scope in distribution,
• marginal costs of bundling are low.
• production set-up costs are high,
• customer acquisition costs are high.
• consumers appreciate the resulting simplification of the purchase decision and benefit from the joint performance of the combined product.

Note also there’s a darker cousin to the above, called Product Tying, in which the consumer is forced to buy the whole set and not just one. This can lead to crappier products becoming more successful, and is the kind of thing you can read about in DOJ monopoly cases.

When bundling is helpful
As mentioned in the list form Wikipedia, there are many situations when bundling is helpful to both the consumer and the business. The bundling is extra helpful when:

• The product being bundled “makes sense” to the consumer
  • “Makes sense” often means a complementary good (drink+burger)
  • Or, it might share the same context (2 of product X are better than 1)
  • Clearly targets the same audience (people who like A also like B)
  • etc.
• Also it can be a great bundle if it was something you were going to buy anyway – like if you put two items in your cart, hesitated and took one out, but were then offered the bundle together

Just as in advertising, you need to “target” your bundles and make sure they are as relevant as possible. If the industry continues to deliver irrelevant offers to consumers, then it’s no surprise that ultimately the whole thing will be written off.

I’m sure I am missing many other examples from above – please write in the comments if you have additional thoughts.

Product bundling in the offers and leadgen world
With the above points in mind, you can imagine what is happening behind the scenes in the leadgen/offers world for social gaming.

The product bundle ends up:

• X dollars worth of virtual currency
• Y dollars worth of bundled product (plus Z dollars of built-in marketing expense)

We can look at this from a couple points of view.

For the product seller, if you’re selling a product for $20, and it costs you $5 to make the item, then you have $15 worth of margin to spend on marketing and still break even. Thus as the product creator, you would be excited about buying up to $15 of virtual currency for the user, if it gets them to buy your product. And if you can buy even less currency for them, that generates profit for you and the leadgen networks and publishers between you and the user.

From the user’s perspective, the above deal can work well if the bundled product “makes sense.” If you were already going to buy a Netflix subscription, and you are being offered the same price and you get some virtual currency to your favorite social game, then that’s great.

So when Michael Arrington of Techcrunch writes that it’s bad for users to pay more for in-game currency than if they paid cash, I think that’s just misunderstanding how offers actually work in the aggregate economy:

In short, these games try to get people to pay cash for in game currency so they can level up faster and have a better overall experience. Which is fine. But for users who won’t pay cash, a wide variety of “offers” are available where they can get in-game currency in exchange for lead gen-type offers. Most of these offers are bad for consumers because it confusingly gets them to pay far more for in-game currency than if they just paid cash (there are notable exceptions, but the scammy stuff tends to crowd out the legitimate offers). And it’s also bad for legitimate advertisers.

How offers solve the 1% problem at Point of Sale
Ultimately, the biggest problem that offers solve for advertisers is the 1% problem of e-commerce. That is, at any given time, the number of people “in market” for anything is actually quite small, and the percentage chance that they will actually purchase something is also very small. As a result, if you are at a “Point of Sale” and they have their credit card out, you might as well try to cross-sell and bundle as much related stuff as possible.

The real skill and value created in all of this, of course, is in actually creating useful product bundles rather than the asinine ones I keep seeing. Social gaming and life insurance don’t mix, the same way that Free iPods and life insurance didn’t mix for incentivized leadgen.

This doesn’t mean that offers companies aren’t totally slimy and the industry isn’t broken
I want to make it clear that all of the above isn’t a judgement on whether the offers industry is working or not working. Frankly, it’s probably pretty broken (I’ll leave that discussion for another post). But I do believe that there is some fundamental value being generated, in the long-run, and someone will build a great company around dynamically creating and targeting product bundles at Point of Sale, wherever you are across the internet.

Whoever does figure that out will make a lot of money, and we’ll forget about all of this social gaming stuff.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

Small profitable companies versus VC-backed startups

I recently had an interesting conversation with a friend centered around a key question that’s come up a couple times before:

How transferable are the skills you learn from building a small, profitable company versus doing a VC-backable startup?

This question came up because part of his life plan was that he wanted to do a “real” shoot-the-moon type startup at some point in his career, but before doing that, he wanted to work on a small profitable company so that he could learn more about the process. We had a discussion around the key assumptions around a plan like that, which centered around the question above.

In general, it’s my belief that most of the knowledge isn’t that transferable, and you are better off just trying to do the VC-backable startup from scratch, rather than deferring that experience. In the worst case, if you fail, you still learn a lot about VC-backable startups and what it takes to succeed. Compare this to building a small, profitable company, where even if you succeed or fail, you may not learn what you wanted to learn.

And of course, it’s a perfectly healthy thing to NOT to want to build a VC-backable company, ever. That is a great idea too :-) But for those who want to have that experience but are deferring it, I would encourage you to try sooner, not later.

VC-backable startups have weird constraints
Ultimately, the core of my beliefs stem from the fact that VC-backable startups have to deal with a number of weird constraints:

• they should grow really fast – people sometimes say ideally hitting $50M in revenue in <5 years
• they should be defensible – ideally having real technology that isn’t easily duplicated
• obviously, you want a great, experienced team – ideally experienced operators or cutting edge technologists
• it’s very centered on SF Bay area and less so on a few other areas (Boston/Seattle/NY/SoCal/Austin)
• early stage is focused on proving things out to get each new round of funding, not on profitability (which is a nice to have)
• etc.

Again, most of the above are nice to haves and they are always on some investor checklist somewhere, and are followed loosely/casually in most cases. Similarly, to get in the game, there are significant “community” effects that kick in too – it’s good to have the right angel investors, because they can help connect you with the right VCs. But angel investors are just random people (albeit random successful people), and they sometimes don’t like to give money to strange people from other cities. So they like to invest locally, and only through people they already know.

So the point on all of the above is, VC-backable companies have all sorts of weird constraints on what you have to be able to do.

Understanding these constraints, and working with them, requires a different mindset than if you are just targeting for profitability.

There’s different constraints on Lifestyle companies, aka Small/profitable companies, aka Passive income companies, aka whatever you want to call them
I think most of the constraints above are pretty silly if the only goal is to build a self-sustaining company that can get profitable and kick off passive income. In those cases, you really don’t need all the constraints above, which really take you down a different path.

In those cases, you could really execute your company anywhere – you don’t have to be in the Bay Area. Rapid growth is both unnecessary, and possibly not desired if new users are creating costs! Instead, you might prefer to charge users upfront, so that you can be sure that you can stay cashflow positive. Similarly, it’s fine to just work with your buddies, or family, or whatever you want – there’s less of a need for them to scale the business quickly, nor will their experience level play a role in whether investors fund the company.

What both the two styles of company do share, however, is that you still need to be able to build a product, and build a business for cheap, even if you are going after different goals.

But even with product development, when you are going for a smaller, self-sustaining company, it’s more OK to target niche markets or build high-quality products for slow-growth businesses. You probably don’t want to build for a new market, since that can take a lot of time and capital to get right.

How much do you really learn?
To net this discussion out, my point is that the two styles of companies are different in as many ways as they are similar. Instead of “walk before you run” it’s more like “learn to sail versus learn to bike.” Learning to sail does not increase your chances of success at cycling, and vice versa, as well.

So for all the engineers out there who are thinking about doing small web projects before trying to take over the world – go for the latter :-)

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

My experience in venture capital

As I’ve blogged about before (though quite a while ago), I spent some time at Mohr Davidow Ventures as Entrepreneur-in-Residence – for more about what that job is, read here and here. A couple years before that, I had spent some time at MDV in their Seattle office, towards the end of the dot com bubble, as an analyst/intern. Both experiences were a ton of fun, and I justified the ~3 years in venture capital where I could have been starting companies as an education that would help me later on.

Now, a couple years later, I thought I would reflect a little bit on where the VC experience helped and hurt me relative to actually trying to build a startup. The net of it is that the time was mostly helpful, and a big chunk of knowledge transferred over, but it was mostly high-level stuff. A lot of running a startup involves mastering nitty gritty details, and the VC experience did nothing to help there :-)

For the lazy/impatient, here are some key things I’d say where it can help:

• It helps with traditional investor/entrepreneur information asymmetries
• Lots of tactical holes still exist
• Investors can often oversimplify startup issues, or overmatch on patterns
• Helps with understanding of investor motivations, which can otherwise appear mysterious

Let’s dive into each of these issues below.

It helps with traditional investor/entrepreneur information asymmetries
Some of the stickiest situation for entrepreneurs are cases where they infrequently encounter a situation, which generates information asymmetries where an investor often knows much more. These asymmetries often involve events like fundraising, selling a company, recruiting executives, etc. In the positive case, investors can be helpful and coach startups through these times, which is great. In the negative case, it provides an opportunity for investors to engage take advantage of naive entrepreneurs, which is not so great. This is why sites likes VentureHacks and TheFunded are useful, because they help even the playing field.

Part of the problem for me, however, is that only the General Partners at VC firms end up actually doing deals. All the associates, EIRs, etc often participate, and you see the final deal terms, but rarely get to see all the back-and-forths that end up with the deal getting done. This creates familiarity with the process, but not the battle-tested experience of having gone through lots of nitty gritty negotiations. But even then, you hear about, and know what the levers are, so everything is less mysterious.

(But like I said, VentureHacks and TheFunded are great, and I only wish there were sites with that level of candor about this obscure industry)

Lots of tactical holes still exist
One area where a venture capital background didn’t help at all was dealing with all the tactical details of getting a company off the ground. In particular, the biggest hole by far is hiring and managing people, which gets abstracted at the financial level. Someone in VC-land can talk abstractly about strong teams, but you don’t have to go through the process of interviewing dozens of people to find the right person.

I’ve written up some of my thoughts here on this topic, in a post called “Building the initial team for seed stage startups” where I talk about a couple points I’ve come to believe:

• Hiring T-shaped people versus specialists
• Try to get doers
• More candidate flow solves a lot of problems
• Interview for the actual work you’ll be doing, not skillset trivia
• Raw intelligence is just one factor – don’t overestimate it

There are also some even deeper questions that are unanswered by VC experience, such as how you actually build out a suitable recruiting pipeline? Or how do you interview people where you don’t have the skillset to comment about their competence one way or the other?

I would say hiring is probably one of the most difficult areas to master, and although there are other block and tackle issues – accounting, leasing an office, operations, etc – getting the right people is just a very hard topic. It’s not a surprise that so many startups struggle with it.

Investors can often oversimplify startup issues, or overmatch on patterns
Venture investors often spread their time across a whole number of industries – you look at their websites, and they’ll say they invest in everything from consumer internet to clean tech to life sciences. MDV was no different, and we were responsive to companies across a large number of markets. One VC explained to me early on that you have to respond to what entrepreneurs are producing, and if you get too “top-down” about a particular industry, it gets easy to overinvest in a bunch of mediocre companies rather than trying intently to just focus on finding the best single team and opportunity you can.

Mike Moritz has talked about this before:

Moritz waxed philosophical by comparing venture capital investing to bird spotting. “I rarely think about big themes. The business is like bird spotting. I don’t try to pick out the flock. Each one is different and I try to find an interestingly complected bird in a flock rather than try to make an observation about an entire flock.” For that reason, while other firms may avoid companies because they perceive a certain investment sector as being overplayed or already mature, Moritz said Sequoia is “careful not to redline neighborhoods”.

Continuing with the ornithological analogy, Moritz pointed to Cisco and said, “There’s a lot to be said for investing in the ugly duckling.” When Don Valentine led Sequoia Capital’s investment in Cisco, many others had passed on the husband and wife founding team of Len Bosack and Sandy Lerner.

One of the difficulties for me personally in seeing a wide variety of companies all the time was that it was impossible to not start to pattern match and draw conclusions about the companies that were probably false. You end up in the proverbial “mile wide, inch deep” level of knowledge about that industry, which makes it all too easy to make generalizations. Similarly, there is a drive to simplify your understanding of a company, since you have to socialize it and talk to other venture partners about particular spaces and companies, which also causes oversimplfication.

Contrast this to startup life, where you end up devoting yourself to one company (which may encapsulate many ideas, as you iterate) for the period of years. You end up diving very deep into situations, and learning about all the different details tradeoffs that cause products to be successful versus not.

Helps with understanding of investor motivations, which can otherwise appear mysterious
Finally, one area where having a venture background helped a lot was understanding investor motivations in general. Entrepreneurs ask a lot of great questions, like, “Why don’t investors want to invest in my idea X which will be highly profitable?” or “Why does hot consumer internet startup X lose tons of money but is valued so much?” The answers to these questions drive a lot of investor behavior, which can be mysterious if you don’t know what’s going on.

The interesting part is understanding why VCs are structured the way they do, why they have a 1 in 10 portfolio strategy, and how they think about their Limited Partners. They have a boss too, of course :-)

The major point here is that building medium-sized, profitable companies that aren’t growing quickly is not really part of the venture capital model. Knowing that can help with all sorts of things, such as massaging your business plan into something “sexy” that investors will respond to. Similarly, it will help get everyone aligned on major decisions, such as financing events, exits, exec team building, etc.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

Welcome to part two of a series of articles from Gambit, a microtransactions platform – you can read the first post here. In the last article, we discussed the average revenues earned from various demographics, and this article touches on implications in product strategy. The author, Susan Su (@susanfsu), is a writer, marketer, and Stanford alum who’s currently at Gambit Payments. Please comment with any questions, and enjoy! –Andrew

Ignore Cougars, Follow the Money: 3 social gaming tips for monetizing younger users
by Susan Su, Gambit Payments

Lately, we’ve all heard a lot about the middle-aged housewife. She’s an adult, she’s got disposable income and a couple of credit cards, probably even a PayPal account. In her leisure time, she sits at home and plays social games with her Facebook friends, possibly instead of going out to a movie. She buys virtual goods with real money while you fill your Olympic-sized swimming pool with gold coins.

This is a great bedtime story to fall asleep to, but would you feel so relaxed if you knew you were leaving millions of dollars on the table?

Age is only one factor
In a previous post, we looked at user age as a factor in a game’s overall revenue. We took a bird’s eye view of average revenue per paying user (ARPPU) by age and transaction volume, and saw that older users – the middle-aged housewife (or husband) – brought in ARPPUs that were 2 or 3 times as much as ARPPUs for younger users. Because of low transaction volumes, however, older users represented little more than a tiny speck of total revenues across the developers in the dataset. It became clear that age data – and even ARPPUs – meant little without the context of volume.

On the other end of the ARPPU spectrum, younger users delivered ARPPUs that were fairly unsexy and unvarying, ranging from $2.58 for 16 and 17 year olds to $3.07 for users aged 20 to 29. However, users aged 14 to 29 together make up 91.5% of the total userbase for the virtual goods industry.

$2.58x…millions
A $2.58 ARPPU doesn’t look so bad when you’re selling to millions of users. The massive transaction volumes associated with teens and 20-somethings aligns directly with this group’s share of total revenue across a sample of nearly 2 million virtual goods users. How massive? Over 93%. That is, users in their teens and twenties bring in over 93% of all revenue seen across all games, for all developers in the sample. Even though ARPPUs are consistently modest, transaction volume – and with it, total revenues – are jaw-dropping.

Younger users are cheap, plentiful, and worth your attention.

Given these figures, what’s a developer to do?

For starters, don’t ignore your younger users. It’s true that there will always be transaction and ARPPU variation based on the game you’re building, what you’re interested in, and what resources you have available, but it’s clear that younger users are still the major players across the board. There are hordes of them, and they’re eager to engage in lots of transactions.

Get Them Young: 3 tips to monetize younger users

1. Think volume. Look for the users who are transacting the most, and then make sure you understand exactly who they are (and how they might be changing). For example, today your revenue may be driven by a massive group of teenagers, but what will happen when those teens become 20-somethings? In this series, we explored this question by age, but you’ll also want to think about geography, language, and gender. ‘Think volume’ means:

• Mind your game. If your product is subpar, you shouldn’t expect amazing volumes or revenues, no matter how much you…
• Focus on growing traffic through virality. How can you make your game even more social, more addictive, and more spreadable?
• Get users to complete. Users are 3 times as likely to make additonal payments if they’ve completed at least one offer.

2. Hold on to your users. People of all ages get tired of games easily. The last thing you need is a poor user experience to push users over the edge and straight into the database of a competitor. Do certain offers just rankle your userbase (leading to poor conversions, bountiful complaints, and churn)? While your payments solution’s algorithms will help you find the best offers for your users, there are always going to be a couple that just don’t perform. ‘Hold on to your users’ means:

• Pick out and remove underperforming offers, either individually or by offer category, and address customer complaints. For example, ‘adult’ offers may not work well if your game’s users are primarily 13-17 year olds.
• Diversify your product(s). How can you enrich a single game to be more complex and engaging? How can you offer more complementary games so when a user defects, she defects to another game in your suite?

3. Keep your eye on empty spaces. Yes, Facebook is huge. Yes, Zynga is dominating. But, growth potential is everywhere still. As more users of all ages sign up for their first Facebook accounts, more people pour into the virtual economy. As Facebook grows in locales outside the U.S., so do the games and apps that inhabit its ecosystem. As users get tired of specific games, they’ll start looking for other places to spend their time and money. They’ll probably invite their friends, too. ‘Keep your eye on empty spaces’ means:

• Don’t make a play just because someone else is making bank off of it (for now). Today’s leaders got there because they kept their eyes on empty spaces and filled them, quickly.
• Look for under-monetized user groups. How well is your game doing with young males? Can you work in a way for more of these users to complete their first offer (and open the door to additional payments)?

These should be your main considerations:

Growth
What does the growth trajectory look like for young users? How many of these users are already playing games, and how many more aren’t? The online casual games industry is still young and has plenty of room for growth.

Facebook boasts 300 million active users, with almost a third of these in the U.S. Since the entire population of the United States is just over 300 million, that means approximately half of all U.S. internet users, or a third of the entire U.S. population, are on Facebook.* Facebook counts 70% of users as having ‘engaged with a Platform application,’ meaning that most users have loaded an app of some sort at some point in their Facebook time. Judging by the impressive monthly active uniques the biggest developers are enjoying (51MM for Zynga’s Farmville alone), it seems that games have already taken off on the network. With all this, is there still room to grow?

Yes. Here’s why:

• Facebook has saturated the U.S. market, but that doesn’t mean every Facebook user is playing a game. Yet.
• The U.S. isn’t the only country in the world, either. In terms of Facebook traffic growth rates, the U.S. doesn’t even make it into the top 10. As other economies (real and virtual) catch up, markets around the world should start looking more and more promising for developers looking to monetize.
• People get tired of games. One developer’s churn is another developer’s new user.

As mentioned above, younger users contribute the lion’s share of total revenue for virtual transactions – for now. However, Facebook reports that the 35 and up group is their fastest growing demographic, so will we see this shift reflected in game usage and monetization too? Probably. But until the older users reach critical mass on the network, would you rather be competing hard for the same handful of housewives or slyly going for the many younger users at lower ARPPUs and massively higher transaction volumes?

Changing ARPPUs
Do ARPPUs change as users get older? Will your 15 year old user be worth more after she turns 18, gets a better job, and starts opting for direct payment over offers? We know that the typical 18 year old makes you more money than the typical 15 year old, so from this we might guess that it will pay off to hold onto that user as she ages.

Older users
Should you try to grow your older userbase? As just mentioned, Facebook’s fastest-growing demographic is the 35 and up set. While actively trying to acquire these users (over others) may divert your resources in ways you can’t afford, it’s likely that your game will indirectly absorb the benefits of Facebook’s demographic growth anyway. If everyone else is focusing on winning the middle-aged housewife segment, would you be better off stealthily (and expertly) acquiring the forgotten younger users? Try it. Measure it. Report back.

Conclusions
In parting, don’t buy into a ‘must do’ (eg. housewives) just because it’s popular today. Popularity doesn’t mean it’s wrong, but it does probably mean that lots of other developers are out there thinking the same thing as you. Instead, look at the data and focus your work where the greatest opportunity currently blossoms. Right now, that’s users who are in their teens and mid-20s.

If you’ve been targeting and you’re seeing interesting results, please share in the comments. What’s worked for you, and what would you do if you were a new developer just entering the marketing today?

For specific questions on data or resources, you can contact Susan here or follow her on Twitter at @susanfsu.

Want more articles?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

Designing a viral loop has multiple stages
Viral loops have been featured in mainstream media and there’s even a book coming out on it – but the step-by-step design of creating a new loop remains obscure, and for good reason. I’ve come to believe that creating viral loops is akin to building a software project – at best, it still comes down to a great team, a strong understanding of the tools available, and relentless iteration. There’s no recipe at the heart of it which guarantees a viral process every time, the same way that you can’t guarantee that any software project will result in market success.

There are no silver bullets in viral marketing
In fact, the core of virality ensures that there will never be a dominant “recipe.” If everyone knows how to build a viral loop around social network invites, then everyone will do it, resulting in consumers will become desensitized, which finally leads to lower response rates. Thus this causes the viral loop to unwind, which leads to long-term disaster.

The only way to combat this is to build a viral loop around the core of your product – something that no one will seek to duplicate, unless they are a direct competitor. These viral loops are incredibly effective because they are lasting and sustainable.

I wanted to jot down a couple thoughts on the different stages that viral loop design go through, so that the entreprenurs reading through this can imagine deeply ingrained, user-aligned ways for their products to gain distribution.

Strategize: Stage 1
The first stage of a viral loop is developing the core strategy around the loop. This requires the viral loop designer to think through, step-by-step, how a user will come to find their product and how they will ultimately pass it along to their friends. If you’re lazy, there are lots of recipes to follow from the Facebook ecosystem like quizzes, “find your friends,” and gifts. As discussed above, these opportunities are already becoming less effective every day.

Even if you decide to use an existing recipe, here are some higher-level strategy questions that should be answered before proceeding:

• How does this viral loop fit into your core product?
• What is the fundamental value proposition you are presenting to your users?
• If your loop is successful, will users transition to your core product or will they bounce when reaching the switchover point?

As you might imagine, most of the discussion here is qualitative and there’s very little A/B testing involved.

Implement: Stage 2
The next stage is the rapid development of the core viral loop. This part should hopefully take days or weeks, not months. It will also certainly be wrong. The best advice I can give here is to follow agile development models and to build the smallest number of features and pages to create the initial flow of pages.

As mentioned before, the best implementations are strongly tied to the core product – as a result, if you’re a video site, it’s best if you can somehow involve videos. If you’re a dating site, you probably want to involve dating.

The other implementation advice I’ll give is to treat the viral loop code as an iterative, protoyping process. So copy and paste all you need, keep it in a separate codebase, and make it easy to refactor. You’ll need to do a lot of messy stuff like changing the order of pages or page elements later, and once you develop your own recipe, it’s easy to rewrite it in the “right way.”

Launch: Stage 3
The next step is to beg, borrow, or steal traffic :-) The easiest way is often to pay for it, $50/day or so, just so you have a trickle of traffic coming in.

Optimize: Stage 4
As you get a flow of incoming traffic, this allows you to deeply optimize the experience. This will involve building out some basic infrastructure to do A/B testing, or using Google Web Optimizer, and otherwise. The key thing here, of course, is to measure whether or not the $50/day you’re spending results in traffic above and beyond what you’re paying for – the more the better, and eventually you’ll cross the threshold where traffic scales infinitely.

In this stage, there are a lot of common fixes that you’ll want to consider:

• Shortening the flow of pages (can you shrink a 5 page funnel down to 2?)
• Rearranging UI elements to emphasize next steps
• Testing different value propositions for going through the flow
• Increasing the # of people invited

This optimization stage creates great conflict for product and customer-oriented people. Oftentimes, to get a number to move from 10% to 30%, there’s temptation to do things that users may not be happy with. This might include things like asking for invites multiple times throughout the initial session, presenting an opt-out process for selecting friends, etc. These are all bad and need to be fixed in order to create a long-term sustainable viral loop.

This optimization step can take a very long time (months is not uncommon) as you zero in on the dozens of small and large changes needed to create a viral loop.

After months of work, two outcomes can result:

• You don’t reach your goal, and you’re stuck on traffic
• You reach your goal, and your traffic is going bananas!

If you don’t reach your goal, then it’s time to stop your optimization process. Often the changes that result are just too small to drive substantial increases in metrics. Instead, you’ll have to rework your entire value proposition, which means to either go back to Stage 2 or possibly Stage 1. This means you’ll want to stop A/B testing and start building out a deeper featureset.

Refine: Stage 5
If your optimization step was successful, your work is probably not done. The final step is polishing your viral loop.

This includes figuring out issues like:

• Making your loop as user-aligned as possible
• Building a pleasant user experience and removing unnecessary flows or page elements
• Refactoring the code to move it from prototype to production
• Integrating it into your core product in a way that makes sense

A lot of people are tempted to skip this polish step, but don’t do it! Skipping this step means that your initial product experience will suck, or be offensive.

In fact, when there’s “excess” virality, that’s a great opportunity to make changes to the viral loop that make it nicer or friendlier. In general, if you are getting exponential growth, it’ll be great even if it’s a slower exponential. What’s more important at that point is spendfing your extra growth towards changes that positively impact long-term retention.

On the other hand, if your product is just meant to be short-term mad money, then by all means skip this step :-)

More on viral loops and marketing
For those that are interested, I’ve written more about viral loops and marketing here.

Today we have the first part of a fantastic two part series where Gambit, a microtransactions platform, is sharing exclusive data and analysis for the payments happening on their platform. The author, Susan Su (@susanfsu), is a writer, marketer, and Stanford alum who’s currently at Gambit Payments. She wants startups to make it big, and you to make more money. Enjoy! –Andrew

Age (and ARPPU) ain’t nothing but a number
by Susan Su, Gambit Payments

In the game of life, you’ve heard that age ain’t nothing but a number. In the world of social games and virtual currencies, the same thing goes. The smart developers know to segment by age groups and target towards those with the highest demonstrated ARPPUs. The even smarter developers know that age ain’t nothing but a number – a single, lonely metric that can dangerously limit your view when you exclude crucial supporting data.

In this post, we’re using demographic data from Gambit Payments to get a bird’s eye view of ARPPUs by age and transaction volume. We’ll see that age data – and even ARPPUs – mean little without the context of volume.

A look at highest grossing ages
Which users pay the most to play?

From this data set, you can see that Gambit’s developers got the highest ARPPUs with users aged 50+. In this month, 60 year old users brought in a $7.92 ARPPU, more than double the ARPPU seen with the younger set.

Players in their 40s averaged a $4.39 ARPPU range – a pretty impressive figure still. Going younger, players in the 30 to 39 year old range brought in a slightly lower ARPPU of $4.11 while players in their 20s brought in $3.07 on average. Finally, teenage players brought in ARPPUs in the mid-$2 range.

This data should come as no surprise. Let’s take a look at the main levers feeding into ARPPU:

• Income. How much money does this user or group of users make? In most respects, this lever is straightforward; if the user in question doesn’t pull in an income, they won’t initiate direct payment for your currency. But, that’s what offers are for. Note, however, that offers typically do not bring in the same flashy ARPPUs as direct credit card or PayPal payments.
• Access to which type of payment. What payment methods are available for this user or group of users? Since we’re talking ARPPUs here, a paying user is a paying user – and thus already has access to some type of payment. However, remember that not all payment methods deliver the same dollar value to your pocket.
  • Does this user or group of users have access to credit card payment or PayPal? If so, you’re in luck. These methods typically bring in the highest revenues because they’re relatively easy and impose minimal friction. Note that PayPal penetration may be low in some parts of the country and world, so it’s unlikely that PayPal will be your biggest breadwinner overall.
  • Will they be paying through their mobile provider? With mobile, money travels through lots of different hands – mobile aggregators, mobile operators, mobile payments providers – before it reaches you, a trickle-down process that will affect earnings accordingly. Also, keep in mind that mobile is based on fixed pricepoints, which gives you less flexibility for what people will pay for. Finally, when paying with mobile, there’s also a cap on a transaction’s dollar amount – you can’t, for example, pay for something costing $100 through your mobile service. While mobile payments typically bring in lower ARPPUs, they also have lower access barriers and are relevant to a wider swathe of your users.
  • Will they be completing offers to earn your currency? Offers can bring in decent ARPPUs, but, for certain user groups, may lack the longetivity of direct payment methods. Will your users complete offers, only to decide that they hate the experience and would rather abandon the process – or your community – altogether? How will you deal with this? For further exploration of this topic, see Gambit’s post on user complaints and coping strategies.
• Willingness to buy online. What is this user’s comfort level with online purchasing? If they’re uncomfortable with online purchasing, ARPPUs associated with this user or group of users will dive accordingly. This becomes a particularly interesting question when you start looking at other demographic data in addition to age – you may find, for example, that users in a certain geographic region are more comfortable with online purchasing because of variance in internet penetration or fluency.

If these levers sound familiar to you, you’re doing well so far. Now let’s see how each of these factors works in the context of the data presented above.

Older users
Older users not only have disposable income, they have access to the payment utilities – credit cards, mobile phones, PayPal accounts – that bring their money to your community. Why 60 year olds specifically? You should view the fact that 60 year olds were at the top as a datapoint specific to this set (an outlier) than a generality that should be extrapolated into rule. If you take a look at the groupings, the 50+ group still achieves an average ARPPU (across individual years) of $5.20 – pretty impressive.

At the other end of the spectrum, your community’s youngest paying participants probably don’t have jobs or the disposable income they bring. Their access to direct payment methods is likely to be highly limited or nonexistent. On the other hand, they probably do have access to mobile payments, and can always complete offers. Based on the notes above, you know that payment via mobile and offers will mean lower ARPPUs for these users.

The key here is to know your users – Who are they? How much money do they make? Where do they live? What types of payment methods are available to them, and how willing / able are they to engage with different methods?

Revenue breakdown by age
Finally, does all this mean it’s time to regroup your acquisition efforts and start to go after the 50+ set (or, if you have been already, give yourself a hearty pat on the back and quit working so hard)? Not yet. Let’s take a look at the percentage of total revenue that these groups bring in, respectively.

It turns out, despite impressive ARPPUs, the 50+ group makes a sad showing when we start looking at percentage of total revenue. If we’d halted our analysis at individual ages, or even broader age groupings, and the ARPPUs they demonstrated in this data set, we would have missed the point entirely.

Transactions breakdown by age
For Gambit developers, 50+ was the goose that never laid its golden egg. All the users in this entire group represent only half a percentage point of Gambit developers’ total revenue for this period. This isn’t because there are 200 age groupings, either. Let’s take a closer look.

Wow. The 50+ group represents a meager 0.3% of total transactions – a figure so small that it barely registers a speck on the revenue radar for Gambit’s developers. Users in their 20s, by contrast, produced 22.5% of all transactions. Finally, teen users represented a whopping 73.5% of transactions made across all Gambit developers. 73.5 versus 0.3… suddenly that $7.92 ARPPU doesn’t seem so significant anymore.

What’s good about the user groups bringing in lower ARPPUs, and how do you optimize their experience to impact your revenues? Conversely, is it possible or worthwhile to improve transaction volume for the highest ARPPU groups? In next week’s post, we’ll go over the strategy implications of the data we presented here and contrast a few approaches to make you more money.

For data geeks
If you prefer to look at the above data in a neat table instead of a fancy pie chart, here it is:

Hope you enjoy this data from Gambit Payments, and part 2 of this article will be coming soon!

[Andrew: Thanks again to Susan for putting together this great post!]

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

A quick rant:

Every time you talk to an ad network or leadgen network or whatever, if you ask what their differentiation is they will say “targeting.” That’s probably wrong, and let me tell you why, based on the recent announcement of the Netflix prize winners:

Netflix was able to wring three years of research to nudge its recommendation algorithm up 10.5 percent, at a cost of $1 million in prize money — a stunning feat on its own.

This means if you combine dozens of the best machine learning people in the world, some of the cleanest datasets, you get a measly 10.5% increase. Compare this to starting a new ad network where you end up with noisy datasets, lots of crappy traffic, and a small team looking at the problem – that’s not an easy path to disruptive change. In general, 10% is not a big enough number to counteract the other economic drivers in the ad market, which revolves around better deal terms, a larger selection of advertisers, better ad inventory, etc.

I would guess that you need a number closer to 50% lift or higher in order for an upstart to dramatically change the ad landscape and neutralize the weapons of the mass of ad network players.

I think disruptive change will come not from algorithms, but rather two other areas:

• Better ad inventory: New websites and mechanics emerge all the time, and who knows what happens when you put ads on them? It was clear, until they tried it, that with the right ads search can be >30% clickthrough rates or more, which is unheard of.
• Better data: The other big opportunity is in using specialized data to drive your algorithms – rather than basing everything off of domains, cookies, and ad impressions like everyone else, there may be ways to extend the targeting to unique datasets that no one has access to. This is what’s happening in the world of retargeting.

The Netflix prize also included people adding in additional data, and that’s factored into the 10.5% improvement. Anyway, the point is, increasing performance on stuff like this is very hard, so when an ad network tells you about their targeting, you should push them instead on their revenue split ;-)

If you don’t watch out, after a couple months of A/B testing, your product will end up looking like Las Vegas!

Why A/B testing and visual design come into conflict
It’s great to implement consistent A/B testing in their product process, but then it becomes even harder to keep a consistent visual design while doing test after test. This tension comes from the fact that A/B tests push you towards local maxima, making the particular section of page you’re testing high-performance, but at the expense of the overall experience. As a result, there’s a lot of temptation to “hack in” a new design, the way that software engineers have to “hack in” a feature – but this is short-term at best. This often means adding a bold, colored link to the top of page with “NEW!” or adding yet another tab – these are all band-aid solutions because once you get to the next set of features, it’s not a scalable design to have 100 tabs.

Each of these competing features, taken by itself, moves the needle positively. However, there isn’t a great way to measure the gradual “tragedy of the commons” effect to the overall user experience. Each new loud page element competes with all previous page elements, and must be louder as a result – this leads to the Vegas effect that many Facebook apps end up in.

To really solve this problem, you need a central design vision – there’s no way around that. It also helps a lot to have a flexible design that embraces A/B testing – you can work with your designers to make this happen through modular, open elements.

Closed designs make it hard to add or remove content
Let’s take a particular example and look at it – this might be a standard example on a page like a video or otherwise:

It looks nice, but also has tremendous sensitivity to the content and an inflexible design that makes it hard to test new content. To be more specific, ask yourself the following quesitons:

• If you wanted to add a comments count in addition to views and votes, how would you do that?
• What happens when the views number gets beyond 10,000?
• What if you wanted to add favorites, or flagging for inappropriate content?
• If we decided to hide the thumbs down, how would this visually balance?
• If we wanted to fit more thumbnails onto a browse page, how easy it is to shrink the main thumbnail?
• etc.

The above design is an example of a “closed” design where everything fits just right, but makes it very difficult to add or remove elements. There’s an exact balancing of all the parts of the element, which makes it very sensitive.

Many of the solutions to the questions involve either require building out new pieces next to the element, which throws it off balance. Thus, if the above were used in an A/B test, the visual look would be immediately ruined.

Open designs that are A/B test-friendly
Let’s compare this to the elements below, which have a more modular design that can scale vertically:

The above elements don’t have the same “just right” visual appeal, but make it much easier to add and remove content. The key design decision is to add multiple bands of content which can be grouped together and extended vertically. Ideally, you would never end up with a repeating tile of 4-buttons and 3-stats, but you could certainly test it much more easily than with the closed design.

Here are some of the variations that can easily be tested:

• Switch the title section and the stats/buttons sections
• Add and remove buttons (or no buttons!)
• Add and remove stats (or no stats!)
• Combine price tags with other stats
• Try different buttons
• etc.

Following an open design on page elements enables substantial A/B testing within some flexible constraints. Now you may still be tempted to do something crazy like big hover overlays, <BLINK> tags, and other stuff, but at least you can make it easy to test a wide variety of low-hanging fruit. It also makes the owner of the overall visual design able to maintain a central “style guide” while still offering enough flexibility to keep people creative.

This same idea of open designs with horizontal bands of content can be applied to whole pages too – let’s examine a page from the king of A/B testing, Amazon.com.

Open page layouts
From the snapshot below, you can see that Amazon groups the center column of content – each element has a title explaining how it is, a list of items, and a navigation link to see more. This is also true with the item detail pages, which use a similar grouping to show everything from similar books to reviews to other elements. These pages can get very long, but because most of it is below-the-fold, it’s easy to get away with.

I’ve been told that this modular design enables Amazon to take a “King of the Hill” approach to testing each horizontal band of content against each other. Different software teams will create different kinds of navigation and recommendation, and if it causes people to click through to buy, then it floats up higher in the page. This systematic A/B testing is much more easily enabled when there’s the design flexibility for that sort of thing.

Here’s a snapshot for a reminder of what this looks like:

While you may argue that Amazon’s design is cluttered and actuallysucks, on the other hand, this approach lets them take a very experimental approach to pushing out features. It makes it very low-cost to implement a new recommendations approach and try it out without needing to figure out how to design it into the UX.

What’s next? Modular user flows?
Of course, if you can take a modular approach to scaling individual page elements or entire pages, the next question is whether you can take this approach to user flows.

I’ve never seen anyone do this, but this is how it might work:

• Any linear user flow is identified in a product (like new registration, payment flow, etc)
• This flow might be 1 page, or broken into N pages
• Similarly, every individual page might have a bunch of fields (like photo, about me, etc.)
• As part of the A/B testing process, you might want to drop a new page (or new fields) into the flow
• Then an optimization process shuffles pages throughout the flow to identify the best page sequence and page-by-page configuration

You might imagine something like this could be a very powerful process as it would allow you to identify whether you should offer a coupon pre-transaction or post-transaction, or on any given page, where an input field should be placed.

For those who want to know more, I have written a bunch more about A/B testing here.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.