Andrew Chen Archives

I was recently invited to lead a session on customer acquisition and viral marketing at the First Round Capital CEO Summit (thanks Josh!). I wanted to share the notes I prepared for the discussion below – hopefully most of them will be self-explanatory.

I’m on blogging break right now, but I may expand the below notes into a series of posts when I have more time. Brb!

—
How to get have sustained viral growth:
– Have a great product (ideally in communication or social content)
– Convert user growth ideas into Excel-based hypotheses and clear user funnels
– Build and track each step of your funnels
– Get an initial stream of traffic (Adwords is OK)
– Optimize until every user is bringing in a new user
Timeline: weeks to months

Getting scientific about user acquisition:
– Start with your laundry list of acquisition ideas
– SEO, tell a friend, Twitter, etc.
– Convert into 2-3 testable hypotheses
– “Buy users for $1, monetize at $5”
– “20% of registered users will import addressbooks, >5 of their friends will register”

Viral loops in SaaS/enterprise
– What things do people share? What tools do they use for communication?
– files, wikis, Outlook, Excel, USB keys, etc.
– These are your viral channels (vs Newsfeed/Notifications on Facebook)
– If your value prop can align with a channel, then you might make it viral
– Case studies: Yousendit, Dropbox, Wikis, Basecamp, etc.

How quick-hit viral loops work for consumer products
– Cialdini’s “Influence: The Psychology of Persuasion”
– Quizzes: Social norms
– Top friends, eCards: Reciprocation
– 8 invites left: Scarcity
– But what’s the followup?
– Hide quoted text –

Value propositions for viral loops
– Best value prop is like Skype
– great for both parties (inviter and invitee)
– build deeply into the product (takes 2 to tango)
– Worst value prop is like lots of FB apps
– little to no value for the inviter/invitee
– lots of churn, feels spammy
– Sustainable viral growth is key for long-term value creation

Different acquisition models work for different kinds of businesses
– Vertical social networks -> SEO/SEM
– SaaS/enterprise -> SEO/SEM
– Consumer/communication/social content -> viral
– Themes, decorations for blogs/profiles -> widgets

Optimize your funnels by brainstorming levers
– Lets say you have funnel of Signup -> Download -> Install -> Fill out profile
– Lots of ways to improve
– change the order of steps
– remove steps
– combine steps
– use lightboxes, or longer pages, or progress bars, or lots of other UI tricks
– To optimize just the download-to-install step, you have dozens of options
– headline
– button placement
– “hero” photo or video
– target their OS
– size of download
– AIR
– small installer vs all-at-once
– installer filename
– etc.

Books and more resources
– Adam Penenberg, “Viral Loop”
– Robert Cialdini, “Influence, The Psychology of Persuasion”
– Tim Ash, “Landing Page Optimization”
– David King and Siqi Chen, “Metrics for Social Games” (Slideshare)
(lots of other resources on Slideshare)

Congrats to my sister Ada Chen, one of the first 10 employees at Mochi Media. I introduced her to Jameson (Mochi’s CEO and co-founder) back when she was first moving down to the Bay Area, and she turned down many other opportunities to go to Mochi. I remember she said she really loved the team, the opportunity, and thought she would learn a lot – which she has. I’m very happy it worked out for her. You can congratulate her at @adachen. (Oh and she’s also getting married this year to @sachinrekhi, another startup guy – congrats on that as well!)

From now, I know people will say, “omg you’re Ada Chen’s brother?” :-) It’ll be great.

When I first moved down to the Bay Area, I originally met Jameson, who was nice enough for me to crash at his old place in the Mission to attend the GDC. He ran Mochi off of a couple tables in his living room, where he lived with Bob in a work/live condo. They’ve gone a long way since then! Congrats to Jameson, Bob, and the rest of the team – well deserved.

Here’s some of the breaking news from the Wall Street Journal:

BEIJING—Chinese online game developer Shanda Games Ltd. agreed to acquire U.S. online game network Mochi Media in a deal valued at $80 million, furthering its global expansion ambitions.

Under the deal, which the companies expect to announce Tuesday, San Francisco-based privately-held Mochi will receive $60 million in cash and $20 million in shares of Shanda Games, a Nasdaq-listed, Shanghai-based company known for creating some of China’s most popular massive multiplayer online games.

…

Also more here at Techcrunch and Paidcontent.

Happy new year! I’ve been reading a ton of great books over the last month, and particularly the holiday break, and wanted to share them below with a couple comments.

Interaction design and rapid prototyping
Recently, I’ve been on a big kick to develop a much stronger opinion about design, particularly interaction design, and to build products prioritizing desirability over a business/metrics/optimization point of view. I’ve recently wrote about this perspective here.

Here are some of the books that have helped me in my thinking:

Inmates are Running the Asylum
This is probably my favorite book that I read all year. Alan Cooper‘s classic book that builds a business case on creating products from a user-centered view rather than business or technology. Introduces the definition of “interaction design” versus other design disciplines, the creation and use of personas, how engineers design software experiences, etc. Really needs to be updated for the agile programming movement, but still a very solid book.

IDEO’s Human-Centered Design Toolkit (PDF)
World-famous design firm IDEO published a toolkit documenting their human-centered design process. It’s longer than it could be because it lists all the methodologies inline, but it’s the deepest look inside IDEO’s design process that I’ve found. The important part is reading about how they go from user research to an insights framework to their “How Might We” questions that drive the creation of many low-fidelity prototypes. I’ve read a ton of books about personas but it wasn’t until I understood this process that I connected the dots on how to go from user research to prototypes to a final product – otherwise, it’s tempting for personas to become a useless artifact that doesn’t drive the product creation process. Read this, but my tip would be to skip through the methodologies on the first read – it’ll make more sense. Also, here’s a related PDF from the Stanford d.school here.

The Design of Business: Why Design Thinking is the Next Competitive Advantage
Artful Making: What Managers Need to Know About How Artists Work
Both of the above books cover similar ground, on how to relate innovation to the broader framework of ideating, designing, deploying, and growing successful products. In Artful Making, the discussion is around “artful” versus “industrial” processes, the former which emphasizes learning by doing and rapid prototyping, versus the factory floor process which emphasizes reliability and efficiency. The Design of Business looks at new product design as the process of moving from “mysteries” (new markets, new ideas) to “heuristics” to “algorithms” to “code” (efficiency-oriented, repeatable processes). The common idea from both books is that new product innovation is very different than metrics-focused efficiency processes, and shouldn’t be treated in the same way. That’s not to say you can’t have a strong, deterministic process around design innovation, but it just requires a different way of thinking.

Serious Play
This book deserves a much longer writeup, since I found it incredibly fascinating. Serious Play is about the notion that spreadsheets are to finance what mockups are to product, and what rehearsals are to theater. They are all models (or, if you prefer, prototypes) that allow people to simulate the future without incurring the full cost of actually doing it. The book touches on many of the first and second degrees of using spreadsheets, clay models, and other artifacts to drive decision-making, including politics, imperfections of models, and what kinds of industries excel at rapid prototyping versus others. Before reading this book, I never really saw the connection between spreadsheets and design mockups, but the author makes a compelling case linking the two as simulation tools.

About Face
Alan Cooper (see above) wrote a more tactical book about the actual “How To” around his Goal-Driven Design process, as mentioned in Inmates are Running the Asylum.

Just for fun
The below books are not necessarily related to startups, but I found them fun and compelling to read.

The Monk and the Riddle
Randy Komisar, a partner at Kleiner Perkins, wrote a philosophical book on life and startups a few years back that I would highly recommend. The core of the book is the idea that too many people try to live what he calls the “Deferred Life Plan,” where you do something you don’t love with the plan to eventually get to your real goals.

Coders at Work
Different profiles of engineers who have worked on important software projects.

The $12 Million Stuffed Shark: The Curious Economics of Contemporary Art
An economist dissects the world of contemporary art, the different players, what drives the economics, etc. I found this interesting from the perspective of art as a virtual good – his view of what causes high prices very much confirms this viewpoint.

Complications: A Surgeon’s Notes on an Imperfect Science
Atul Gawande provides a deeper perspective on what medicine is really like – the mistakes, the uncertainty – all the things you don’t really want to hear as a patient :-)

I also have an older book list here.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

Here’s a quickie roundup of the top posts from my blog over the last year, sorted by pageview. They are heavily skewed towards articles passed on to first time readers since most of my readership is via RSS.

A large number of them related to freemium, which tells you how much interest there was in making money in 2009 :-) Perhaps with the economy returning, there will be a shift of interest towards growth again.

Enjoy.

1. How to create a profitable Freemium startup (spreadsheet model included!)
2. Built to Fail: How companies like Google, IDEO, and 37signals build failure-tolerant systems for anything!
3. Free to Freemium: 5 lessons learned from YouSendIt.com
4. Product design debt versus Technical debt
5. Friends versus Followers: Twitter’s elegant design for grouping contacts
6. 5 warning signs: Does A/B testing lead to crappy products?
7. Freemium business model case study: AdultFriendFinder ARPU, churn, and conversion rates
8. Which startup’s collapse will end the Web 2.0 era?
9. 2009 conference schedule for the digital media industry
10. Does every startup need a Steve Jobs?
11. Why low-fidelity prototyping kicks butt for customer-driven design
12. What if interviews poorly predict job performance? What if dating poorly predicts marital happiness?
13. How to calculate cost-per-acquisition for startups relying on freemium, subscription, or virtual items biz models
14. 5 crucial stages in designing your viral loop
15. Age (and ARPPU) ain’t nothing but a number: Data on how age impacts social gaming monetization

To all my subscribers, thank you for reading!

Happy holidays everyone! I just wanted to make a brief return from a blogging vacation to show you a new Google search test where I’ve been randomly been assigned to the A/B test.

To summarize the main differences:

• Big blue buttons for everything
• Drill-down sidebar after a search
• Emphasis on filtering by time – so you can get the “latest”
• Search across their properties, including News, Blogs, Books, Forums, Shopping, etc.
• Features I haven’t seen (except in labs?) such as Timeline, Related Searches, Wonder wheel, etc.

Really a ton of changes!

Here are the photos: First, the homepage…

And here’s a search results page after an egosurf:

Here’s the expanded sidebar:

There are lots of changes, you can check out all the screencaps below:

UPDATE: Interesting – I’m noticing that the sidebar is switching between all text vs icons + large text, on a page-by-page basis. Plus they are changing the content around by quite a bit. Seems like they are still testing the exact nature of the sidebar.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

I had a recent quote in the NYT article on the Fitbit, and wanted to give a couple quick thoughts about using the device so far. (This isn’t a gadget blog so posts like this will be far and few between).

In general, I love the form factor and the fact I can clip it on and pretty much ignore it for the rest of the day. People that I show it to always remark on how small and cute it is, and I’ve gotten several tweets on how they’re jealous that I have on already. One funny thing is that I find myself checking it absent-mindedly the same why I check FML on my iPhone. In general, it has made me much more aware of my incredibly sedentary lifestyle, and the daily goal of 10,000 steps is a tough one to hit. I’m usually hovering around 5-6,000 steps at most, and have to actively work to get to 10,000.

The web integration is nice although I don’t find myself checking it that often. They just recently added some social features to it, and I’m sure when my friends, and family get these devices it will be fun to see how I am doing relative to them.

Anyway, it’s a basic device and does exactly what it’s supposed to do really well. I’m excited to see when it gets more gadgety and does more with the data.

For a more detailed review, check out the Engadget review or the CNet review.

What’s a minimum “test” of your product? And what are you testing?
A hypothesis-driven approach to product development dictates that you build as much as you need to test our your product, but not more and not less. But what are you “testing” your product for?

One possibility, as lean startups guru Eric Ries has stated, is to test your product for “viability.” He’s coined an important term, called Minimum Viable Product, and I’ll excerpt his excellent blog post below:

The idea of minimum viable product is useful because you can basically say: our vision is to build a product that solves this core problem for customers and we think that for the people who are early adopters for this kind of solution, they will be the most forgiving. And they will fill in their minds the features that aren’t quite there if we give them the core, tent-pole features that point the direction of where we’re trying to go.

So, the minimum viable product is that product which has just those features (and no more) that allows you to ship a product that resonates with early adopters; some of whom will pay you money or give you feedback.

He goes on to state that another example of this idea would be to set up a landing page and test for clickthrough rates and signup conversions, to see if there is any interest in the product. You could also stick a priced offer on the landing page to see how that affects peoples’ interest in registering for the site.

Viability is certainly one bar you can test for, but a related (and overlapping concept) is around testing product desirability. Let’s discuss this further.

Viable versus Desirable
In a previous post, I discussed an IDEO framework for how to think about desirability (user-focused) versus viability (business) and feasibility (engineering) – you can read that post here, called Does every startup need a Steve Jobs?

The idea here is that different companies often pursue products with different primary lenses – a business-driven company might try to assess viability upfront, thinking about metrics and revenue and market sizes. A feasibility (engineering) oriented organization might try to pick a super hard technology first (P2P! Mapreduce! Search!), then try to build a business around it. And a desirability-focused team might focus first and foremost on the target customer, their context and behavior, and build a product experience around that.

Thus, a Minimum Viable Product tends to center around the business perspective – what’s the minimum product I have to build in order to figure out whether or not I have a business? You might do that from testing signups on landing pages, try to sell products before they exist, etc. Putting up price points and collecting payment info is encouraged, because it helps assess the true viability of a product.

But what if you come from a human-centered perspective, and you want to build the Minimum Desirable Product? I think this is a subtle difference with big implications. A minimum desirable product (MDP) would focus primarily on whether or not you are providing an insanely great product experience and creating value for the end user.

Let’s define it as such:

Minimum Desirable Product is the simplest experience necessary to prove out a high-value, satisfying product experience for users

(independent of business viability)

To build an MDP, you will have to actually deliver the core of a product experience so that your customers can make a full assessment, rather than simply providing a landing page. Instead of measuring YOUR conversion rates and revenue generated, instead you might figure out the metrics of what benefits you are providing to the user. (I wrote about Benefit-Driven Metrics a while back) Similarly, you might make extensive use of qualitative research techniques such as the ones detailed by IDEO’s methodology card deck.

This also relates very much to Marc Andreessen’s definition of product/market fit, which he defines in purely market “pull” terms and not based on business ideas or viability. You could view the the Minimum Desirable Product as the simplest product that has a credible shot at providing that product/market fit.

Examples of MVP versus MDP
Let me make some quick distinctions about sites that might be Minimum Viable Products, but perhaps not Minimum Desirable Products, and vice versa.

• If you build a really viral social network that is profitable but has terrible user churn – you have built an MVP but not an MDP.
• If your profitable dating site gets lots of users to buy subscriptions at $20/month, but none of them find hot dates they were promised, you have built an MVP but not a MDP.
• If you build a magic box that spits out money whenever you hit a button, that is certainly desirable but not viable at all.
• If you create an amazing board game that your friends and family love and are addicted to, but you can’t get a game company to distribute it, you have created an MDP but not an MVP.
• If you have created a website with 20M+ uniques/month where people can tell each other what kind of sandwich they are eating, that has probably passed the desirability test but not the viability test.

(btw, I am writing this blog while drinking a soy latte at Cafe Epi in Palo Alto, but not eating a sandwich, for those who are curious)

Is desirability more important for consumer internet startups?
One of the key reasons why I began to think of this question is that it strikes me that consumer internet companies often don’t care much whether or not they have viable businesses in the short run. If you are building a large, viral, ad-support consumer internet property, you just want to go big! As soon as possible! This is particularly true for ad-supported sites where you need to break through a certain size to start talking to the brand ad agencies who can pay up on CPM. (More on that here) As a result of that, the goal becomes to hit product/market fit as soon as possible, and figure out the business model later.

Similarly, the key risk for consumer internet startups tends not to be technical risk or execution risk – it tends to be market risk. That risk may manifest itself as questions on whether or not there’s enough consumer value, or whether or not the market is big enough. These are things that may be proven purely based on desirability-oriented questions rather than getting into the business or technical side at all.

Minimum Feasible  Product?
The last though I will leave you with is, perhaps there are markets where the engineering portion is the most important – and thus the most important concept of Minimum Feasible Product.

For example, for a drug company curing cancer, the focus wouldn’t be on minimum viable product because if you have a cure for cancer, you’ll be viable. Similarly, you may not focus on desirability, because your product would clearly have pull from the market. You don’t need to do landing pages or user-centered research to figure out that curing cancer is a big deal from a business and user point of view.

Instead, the focus would be on Minimum Feasible Product – what is the smallest amount of work necessary to field a credible candidate for an “in lab” solution to the product?

For consumer internet, perhaps there are similar examples of this.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

Found this link and wanted to share it – thought it was an interesting argument. Quoted from Flurry, an iPhone analytics provider’s newsletter:

As all industry eyes look to the iPhone, the iPod Touch is quietly building a loyal base among the next generation of iPhone users, positioning Apple to corner the smartphone market not only today, but also tomorrow. In terms of Life Stage Marketing, the practice of appealing to different age-based segments, Apple is using the iPod Touch to build loyalty with pre-teens and teens, even before they have their own phones (think: McDonalds’ Happy Meal marketing strategy).

When today’s young iPod Touch users age by five years, they will already have iTunes accounts, saved personal contacts to their iPod Touch devices, purchased hundreds of apps and songs, and mastered the iPhone OS user interface. This translates into loyalty and switching costs, allowing Apple to seamlessly “graduate” young users from the iPod Touch to the iPhone.

An interesting thought, for sure.

Read more here.

David Shen, an Apple alum and now prolific angel investor, wrote me to chime in on my recent post on Steve Jobs.

UPDATE: I also heard from Kristee Rosendahl, who co-founded Apple’s Human Interface Group and worked directly on Hypercard, and posted her reply below as well.

I reposted David’s message below, with his permission, where he discusses the indirect effect that Steve Jobs has on the Apple design culture. He says Apple is still ruled by the business and engineering guys, but that his indirect effect is providing a central design vision as well as removing the politics around product design.

David writes:

great post, but i actually have a different viewpoint.

have you ever worked in a big org like apple? it’s filled with competing viewpoints, and is always run by business guys, never design guys. always design guys are relegated very far down the chain, and so thus engineering and business seem to drive the day on any decisions. this is often where we find “fake desirability”. [ed: fake desirability, which he defines “by this i mean that some people said they designed for users, but in actuality they only designed for themselves.”]

when i was at apple, it was certainly better than other companies. but still it was a guy who used to be at IBM germany who was CEO at the time from 1990-1993, after john sculley was removed. and design reported still a level or two below the CEO, but lucky for apple the culture itself supported UX and its products were consistently better.

when steve jobs came, he killed the political bullshit that made great products even better. everything runs through him and if he doesn’t like it, it’s too bad. so you have to suck it up to work at apple, doing steve’s bidding or else you will not survive in there.

he is a design dictator of the company. and it’s fortunate for apple and the world in general, that they have him because without his ironhand, the company would soon devolve back into a political, consensus driven company. it would still have great products from a certain point of view, but i doubt that they would ever have the game changing, superiority they exhibit now. committees would grow, politics would ensue, control battles would happen, and superior products would be hampered by all this. steve removes all that; he makes the final decision and pushes details that no one else would have the authority to push. and being at the top, you have to listen to him or else you’re fired. that’s it; end of story.

and thank god he is right most of the time.

so i would argue that benevolent dictatorships are the best form of govt in the world, including both for companies and for countries, where one person has both the right vision and the ironhand/cut-thru-the-bullshit attitude and style to do the right thing. think if obama ruled the US like steve jobs. he would just do the right thing, and nobody could do a single thing about it.

the probability of another steve jobs occurring is vanishingly small. i doubt that another startup could produce a steve jobs. it is a combination of intelligence, market savvy, strong personality, and ruthlessness that makes him successful. not many people can exhibit all those qualities to make it work.

believe me i have seen people try. but they just end up pissing everyone off and they fail when nobody can work for them, or they think they have supreme market savvy but really they are exhibiting “fake desirability”. remember that steve took decades to develop his ability to this day; a 20-30 year old is very very unlikely to have enough world experience to be able to match that. so maybe you could say that zuckerberg or larry/sergey are in that camp. but there are other tons of people out there who are not. so the probability of finding someone like that (or being someone like that) is pretty darn small….

Very interesting…

UPDATE: David added some additional thoughts in the comments.

I should clarify that I don’t think that Apple is run by business and engineering guys solely now. I think it’s probably one of the most balanced orgs, power-wise in any corporation I’ve seen. What other companies have their head of design reporting into their CEO? I can’t think of any!

But CTOs almost always report into the CEO, and certainly the business structures, like business units, general managers, etc. always do.

Authority and importance are often driven by how high in the org chart you are. Having a voice at the table as high up as possible means you get to be heard and your issues taken seriously, and your influence felt. It also means that the CEO has now told the company: “the guys who report to me are also the most important to me. That’s why they report to me.” If the design lead does not report to the CEO, then how can design truly have a voice in the strategic decisions of the company? It could only be translated through the voice of his manager, and so on, upwards until they get muddied and washed out by the time they reach the top…or just lost.

A clarifying point about indirect design influence: 

I actually think Steve has a direct influence on design on many products, and that the effect of this design influence creates what I’ll call “design philosophy inertia” which propagates through the org, across product lines and down product lines. This is where his indirect influence can be felt. But it is clear to me there are products that he cares most about, and these he will put his attention on all the time.

As I said in the post above, thank god we have Steve. I doubt we’d see the world be filled with such superior products without him.

UPDATE: Some thoughts from Kristee Rosendahl below on Apple and what startups can (and can’t) take away from process.

My comments about Apple have to be taken in the context of when I was there 1984-1990, as things continue to change there like any company. After 1990, I’ve been an external observer of Apple’s culture, just like the rest of us.

I think Steve is a design dictator when it comes to the products close to his heart. The good news is that his approach and sensibility is so baked into the culture of Apple that everyone inside Apple considers themselves design advocates. So other products get the advantage of that. It’s an amazing example of how leaders set tone, culture, and priorities. When I went to work for Apple, even as a consultant first, they gave me this little cubical with a Mac. Then the person said be prepared for Steve to walk in at any time and ask you what you are doing. The implication is that I better be able to defend my work at any moment. That set a tone from day one! He never showed up in my office, but talk about creating an environment based on that.

I also think Steve is in his own class, because he is not only a designer, he is an incredible marketer. I agree with you though that there is still lots of room to improve and elevate design within an organization. The issue will be that most CEOs can’t really talk about design. There are almost no classes in biz school that really address design – I sure hope that changes. So most biz or tech guys running the show are not apt to go there, its not their language, and not their safe zone. One of the major ways I have seen companies overcome this is with two partners as head – one who handles the biz side but totally appreciates and respects design, and the other is the creative lead who has respect and can partner with a business oriented person. The other option is to hire a really good design lead. Mostly, though, companies hire consultants, or agencies. When the job is done, there is no one in house to keep advocating from the top…design has got to be on the executive team and by the water cooler to make it work.

[…]

I would add that, in this discussion about design, remember Apple is a consumer products company. Most of what they are lauded for is their product design, ease of use, delight, coolness, etc. Designing real products people carry with them, work on, and use for entertainment purposes, is a far different design effort than creating a social media website. While both require design, their development time, designer’s skill sets and to-market time are not similar. Sometimes we need to make that distinction when we talk about design efforts in various different kinds of companies and start ups.

When a CEO who is starting up an online business says they want their product ” to be as simple as Apple”, we all know what that means. What start ups forget is how many people’s efforts and hours go into making Apple’s products that clean and simple. In my experience, it has been a real challenge to convey how much longer a simple solution takes over a complex one. A truly simple and elegant solution just demands more time and cycles than most people understand. So I’m delighted when you can hear designers talk about their process and the timeline. A simple product demands patience, lots of iterations and hence, additional expenditures.

At the same time, I’ve unfortunately seen small companies and many startups waste thousands of dollars and person hours spinning about the design of the product because they don’t have a clear idea of the core benefit. So in the end, they could have spent the same amount of money but had a very different outcome – a much better product. They need to get better at doing their homework… see attached Seth Godin post.

This is what Jobs understands and why removing the corporate bs is so important. The company politics or personal aesthetics can take down a good idea or product in no time, even in a tiny company. David also talks about “design by consensus” and I think that’s part of any startup. The group is typically so small, that to leave someone out of the design process early on doesn’t emphasize the “team” spirit of the start up. This can be a big mistake. Not everyone is involved in other parts of the processes – I don’t critique code, for instance. I leave that up to people who are experts at that function. But many people want or think that being part of the design decisions is part of their inherited right as an early team member – it’s fun, distracting and everyone has an opinion. My advice for a startup is to be very careful about how the process is handled. As a designer, this is part of my role as well – to design the process by which this can all happen smoothly. In the end you can get a mediocre design by consensus that looks cool to the internal team but does nothing for the potential customer.

A product’s design success also depends on whether you perceive design as merely a decorative skinning of the product once its developed or as an inherent part of the product development process. I get calls all the time from companies who are launching in 8 weeks, the product is in development, and they need a designer to come in to apply some look and feel to it. This is the antithesis of how Jobs works. And it shows. And it impacts the financial success of the product.

I think we designers also need to keep doing a better job at being part of the development teams. I’ve seen many a designer complain about having to attend development meetings – they just want wireframes and then they can do their magic. I think this is partially why developers have taken on some design roles. And I want to say here, I consider developers designers in their own right! Someone has to make choices early on, and if a designer isn’t there, the product gets developed either way. Designers need to get more agile, iterative, and more transparent in what they do. Today’s products demand that of us.

Lastly, I’m including my absolutely favorite post from Seth Godin. I think it sums up so well many points that would help both startups and existing businesses get a little shot of that Apple DNA. Seth’s observations are a good summary about how equally important fostering innovation is vs being an innovator. Steve Jobs does both pretty well. For now, pick one role and do it really well.

[Seth’s post on How to be a great client]

Thank you Kristee!

What does Steve Jobs really do for Apple?
I had a recent conversation on Apple’s incredible design culture and what it would take to create that in a startup. In many ways, it seems like an insurmountably difficult challenge to play the role of Steve Jobs, with his god-like sense of product aesthetics and interactions.

And yet, Apple has hundreds of products and experiences – hardware, software, HR materials, commercials, etc. Steve Jobs certainly doesn’t have time to work on the design of every Apple product, and of course has 35,000 employees to manage. So what does Steve Jobs really do, to create the amazing design culture at Apple?

And more importantly, can a startup hope to even start to capture the same kind of culture?

Well, let me give you my best guess :-)

IDEO’s product framework for Desirability, Feasibility, and Viability
First, let’s take a quick detour and talk about IDEO’s perspective on new product development – this is documented as part of their 100+ PDF on human centered design, but also recounted to me by a friend who works there.

The idea is that all products ultimately come from an epic struggle between three perspectives: Desirability, Feasibility, and Viability. IDEO focuses on new products from the desirability side, which means they think about how to make sexy products with clear value propositions, and think technology and business goals flow from that. Most of their Fortune 500 clients do not act this way, of course, which is why they have to hire IDEO.

Here’s the diagram included in their HCD toolkit:

The way this was retold to me is that these factors map into functional parts of a business:

• Viability = Business focus (marketing, finance)
• Feasibility = Engineering focus (technologies, agile process, etc.)
• Desirability = Design focus (customers, aesthetics, etc.)

Business-focused product perspective: Viability
For business-oriented products, the focus might be on any of the following:

• “hot markets”
• making money
• funding potential
• distribution
• metrics

The idea there is that you get to a product via one of these first-order items. A business-oriented entrepreneur might identify a market, then try to come up with a product within the market – for example, “wow, Zynga is making $250M/year, and fish games are big. I should come up with a social gaming product too.”

I would also argue that “corporate” thinking (including MBAs and biz plan competitions) fundamentally revolve around this approach – the most important thing becomes the analytical discussion around the business, rather than the core user experience itself. Financial metrics and market sizes become the dominating point of discussion – I would argue also that most venture capitalists fall into this bucket.

The big “religions” in this perspective are frameworks like Built to Last, Crossing the Chasm, Customer Development, Blue Ocean Strategy, even Efficient Market Hypothesis. You might also count Six Sigma, all the stuff in McKinsey quarterlies, etc.

Engineering-focused product perspective: Feasibility
For technology-oriented products, the focus might be on the following:

• programming language and development stack
• cool technologies or libraries
• engineering processes (agile or otherwise)

For people who use this as a first-order filter, you might end up with a line of thinking like, “BitTorrent is really cool, how do we build a business around it?”

I would also put engineering processes like agile into this, because that can easily become a first-order item in how to build a product as well. Agile won’t work for every team, for every product, in every situation, and yet it’s viewed as an all-purpose hammer – does that really make sense?

The big “religions” in this perspective are frameworks are agile, scrum, open source, etc. I might also count the “ecosystems” like Rails as a unique culture with its own set of beliefs and conventions. Frameworks like “Lean Startups” ultimately combine both Business and Engineering goals, via Customer Development plus Agile.

Design-focused product perspective: Desirability
For design-focused products, the focus might be on:

• context, culture, and goals
• customer goals and product experience
• design aesthetics and interactions

The first-order filter in this case might be “Sick people go to hospitals and have a terrible experience – how do we improve that?” The tools employed at this initial stage might include user research, development of personas and user goals, and rapid prototyping to explore many product concepts.

The big “religions” here are led by Apple and their aesthetics and standards. And of course folks like IDEO and their “design thinking” ideas.

How business and engineering goals encroach on the desirability of a product
Reading through the above, perhaps you have identified yourself as prioritizing one versus the other. And in general, the prioritization of the three different goals drives what kinds of product experiences you can build.

From the perspective of making a sexy, highly desirable product, you’ll find lots of objections from business or engineering:

• “spending money on visual design is too expensive”
• “polishing a product will make the process too slow”
• “this product is boring to implement”
• “can you redesign this product so we can build it in 1 week sprints?”
• “this target user is great, but we want the product to be more powerful and support more audiences”
• “but Zynga doesn’t do this, can you just copy them?”
• “why build so many prototypes that get thrown away? That’s costly and slow”
• “if you added X to this product, it would put us into strategic market Y”
• etc.

How do you handle questions like the above?

All of them are great questions, and of course the right answer means you have to find a balance in the approach. But what is the expense towards the core of your product experience?

Back to Steve Jobs – what does he really do?
Long story short, my hypothesis is that Steve Jobs is one of the rare CEOs who is very focused on product desirability. In battles with the business and technology goals, desirability will almost always win out.

So his role isn’t that of a designer, but rather Chief Design Advocate. This means:

• he makes it clear that products should be “insanely great”
• he recruits a top design team, and protects them from competing goals
• he is willing to spend money, adjust technology processes, all for the goal of highly desirable products
• he convinces financial analysts, industry pundits, etc. that product design is very important

To me, the amazing part about this is: Any company can do it.

Maybe not as good as Jobs, but they can decide to make it a priority – but few companies do. With the pressure of quarterly earnings, what competitors are doing, and employee aspirational desires, the focus moves off of killer experiences for customers – that’s no good.

If the above is true, then any of us can be the Steve Jobs of our team. Start by prioritizing design and desirability, and place it on a better footing relative to engineering and business goals. Learn the tools, develop your own religion, and start building great product experiences.

It almost sounds so easy!

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

I have been casually lurking on a new Google group focusing on the techniques around Lean Startups pioneered by Steve Blank and Eric Ries. Lots of fun conversations happening there.

Go here if you want to check out some of the threads going on. I’ve been following via the RSS feeds of new topics.

Here’s the info from the main site – thought I would give it a plug. The guy who runs it is named Rich Collins.

Let’s build a community focused on learning from lean startups.

I’ve recently been captivated by the ideas introduced by Steve Blank and Eric Ries. I’ve been reading their articles, watching their talks and listening to their podcasts. What I haven’t found is a community of other startup founders with a similar interest in lean startup strategies and tactics.

I would like to create a community centered around building lean startups. There will be a website with a forum, wiki, chat and Hacker News style social bookmarking. The focus will be on sharing battle stories and numbers from actual startup campaigns. I will also organize events for members to attend.

In true lean startup form, I created this website to gauge the interest in the formation of such a community. After 24 hours of having the site up, 131 people showed their interest by submitting their email address. As a result, I’ve created a Google Group where we can start the conversation.

Have fun! Hopefully there will be some interesting stuff to come out of it.

Amazon’s tabs are a classic example of product design debt and the refactoring process to pay it down

Incrementalism creates Technical Debt, and also Product Design Debt
Most startups these days build products using the various philosophies of agile – both in the formal sense but also the informal sayings of “deploy early and often,” “fail fast,” “ship and iterate,” etc. Coupled with A/B testing, customer development, and thinking through business problems in a scientific, hypothesis-driven way, you end up with a powerful cocktail of techniques to build a modern startup in the most iterative way possible. This kind of incrementalism is mostly great, and people should generally do more of it.

The interesting part is when you get a couple months into your product cycle. You often end up with lots of half-done experiments lying around, an infrastructure that isn’t built to scale, and a mishmash of code that needs to be refactored. Most engineers know that in this kind of a case, the best practice is NOT to rewrite your code, but rather refactor it continually and take down the so-called “Technical debt” so that it’s always under control.

However, there’s the other side of the coin, which is the product design. After you’ve added a ton of new features and stuck them all on the homepage, you create Product Design debt. The Amazon tabs at the top are a great example of this – you have a design philosophy built around tabs, you scale it as far as you can, and then you have to refactor your design.

Arguably, MySpace is a company that never paid down their product design debt, and their traffic has been impacted as a result.

Anyway, let’s dive into this topic more, starting with technical debt.

Technical debt
Most of my readers are probably familiar with the concept of technical debt, but just to re-summarize from this great article on the topic:

The first kind of technical debt is the kind that is incurred unintentionally. For example, a design approach just turns out to be error-prone or a junior programmer just writes bad code. This technical debt is the non-strategic result of doing a poor job. In some cases, this kind of debt can be incurred unknowingly, for example, your company might acquire a company that has accumulated significant technical debt that you don’t identify until after the acquisition. Sometimes, ironically, this debt can be created when a team stumbles in its efforts to rewrite a debt-laden platform and inadvertently creates more debt. We’ll call this general category of debt Type I.

The second kind of technical debt is the kind that is incurred intentionally. This commonly occurs when an organization makes a conscious decision to optimize for the present rather than for the future. “If we don’t get this release done on time, there won’t be a next release” is a common refrain—and often a compelling one. This leads to decisions like, “We don’t have time to reconcile these two databases, so we’ll write some glue code that keeps them synchronized for now and reconcile them after we ship.” Or “We have some code written by a contractor that doesn’t follow our coding standards; we’ll clean that up later.” Or “We didn’t have time to write all the unit tests for the code we wrote the last 2 months of the project. We’ll right those tests after the release.” (We’ll call this Type II.)

Of course, we are mostly interested in the second type. Eric Ries has a great article advocating for why it’s OK to Embrace Technical Debt. Another great article is from Joel on Software called Duct Tape Programmer. All of these articles are worth reading.

I won’t focus too much on the definition since those other posts do such a great job – instead, I think it’s worth talking about why an iterative approach tends to produce technical debt. I don’t think it happens all the time, but there’s always a temptation for it to happen.

Ultimately, the problem is that if you are trying to learn something about the business, and your technology is meant just to support that experiment, 99% of the time it’s not worth it to do things the “right way.” The reason is that you don’t know if something is going to work, and as a result, you don’t want to invest in scale or perturbing your entire codebase for something that might be disposable. So instead, you just put a 10% or 25% version of the product out there (now commonly referred to as the Minimum Viable Product) and do as little coding as possible to get there.

The problem is, when the feature is successful, very rarely is a team going to then go back and rewrite it – every experiment creates more questions, and the temptation is to move on to the next question.

Product design debt
A similar problem to this is Product Design debt, which impacts the user experience rather than the underlying technology. The same temptations that lead to technical debt also lead to product design debt, because it’s always harder to do things the “right way” and it’s almost never a rational investment of resources. Show me a site that has great visual appeal, and I’ll guarantee that they don’t A/B test.

Product design debt happens because of scenarios like the following:

• “I want to test this new feature, where should we put it? How about the tabs?”
• “Can we throw this experiment on the homepage and see if people click on it?”
• “Our navigation is kind of getting out of control, but if we fix it, most of the site’s features will lose a ton of traffic”
• “We just added a Lists feature and we want to promote it, can we just add a button next to everyone’s name?”
• “Yahoo just bought our startup and they are going to stick us on their homepage!”

(just kidding on the last one)

The point is, as a product experience grows deeper, at some point the initial design philosophy of just adding more links to a page or more tabs or more buttons just stops scaling. Yet it’s often hard to reorganize the whole site, especially if it means taking a short-term dip on traffic, so the “safe” thing to do becomes to incrementally add things until the user experience is horrible.

Kudos to Facebook for looking at their product and deciding that they needed to refactor everything first into a big newsfeed stream of “stuff,” and then all their features into a generic container of apps. They’ve also done a lot to actually remove options from the menu and navigation.

Why homepages becomes a Las Vegas visual experience
Incrementally-developed UIs that are never refactored often turn into a Las Vegas visual experience over time. Ya know, something that looks like this:

Why does Vegas look this way? I’d speculate that all these buildings are ultimately infringing on the public good of aesthetics, and light pollution becomes a tragedy of the commons. If all of those buildings were to power down, it may be that the relative distribution of business would remain the same, but we’ll never know since that will never happen :-)

Important navigational areas like homepages, inboxes, notifications, etc are all the same way. Each incremental menu item is not a big deal, and provides a lot of value downstream, but a slight incremental cost. But do this enough times, and you’ll start to pollute the overall design aesthetic, which is a public good that all features share.

For startups, this shouldn’t be a huge problem because you should have a product person who manages the whole experience and can resolve the public good problem. But there’s a danger in bottoms-up startup cultures where anyone can throw up an A/B experiment, which on one hand is great, but on the other hand creates UX pollution. The other class of cultures where this becomes a problem is short-term optimizing cultures, which may have a “feature of the week!” they want to focus on, which they need to exaggerate each feature each week.

For established companies with multiple teams competing with each other, this may become a key problem because then it really is a public good within the company.

Product types that are most susceptible to design debt
Ultimately, I think product experiences that provide a million little features are the ones that need to watch out the most.

This means:

• Social networking and Community sites that want to unify chat, forums, polls, videos, blogs, etc.
• Portals that want to unify news, communication, tools, etc.
• Games that want to unify lots of different missions, communication, characters, revenue-generating activities
• Retail products that want to unify lots of product categories and SKUs
• Classifieds sites that want to sell lots of different services, products, people, etc

All of the above products are hard to design for because they are meant to be open and support lots of diverse activities, but refactoring the UI constantly becomes a strong need as the initial navigation paradigms probably will not scale.

I wrote an article a while back specifically on social community sites, called Social Design Explosion.

Ideas for when and how to pay down product design debt?
For entrepreneurs out there who are building metrics-driven products but also committed to a great user experience, I would love to hear when and how you pay down the product design debt. Please comment!

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

Upfront design and agile don’t mix well
It’s an interesting problem to try and mix traditional design tasks – visual polish, user research testing, etc. – to an agile development process. A weekly development cycle doesn’t leave much room for several iterations of mockups, the immense effort of recruiting and interviewing users, and all these other important tasks.

Anyway, I was sent this recent link I’d encourage you to read on 12 emerging best practices for adding UX work to Agile development.

Here are the list of 12:

1. Drive: UX practitioners are part of the customer or product owner team
2. Research, model, and design up front – but only just enough
3. Chunk your design work
4. Use parallel track development to work ahead, and follow behind
5. Buy design time with complex engineering stories
6. Cultivate a user validation group for use for continuous user validation
7. Schedule continuous user research in a separate track from development
8. Leverage user time for multiple activities
9. Use RITE to iterate UI before development
10. Prototype in low fidelity
11. Treat prototype as specification
12. Become a design facilitator

In general, the best practices are about taking the down the level of fidelity in the design process and trying to work ahead of the engineers so that they get the fast feedback they need. Definitely worth reading.

Seattle is a great tech city
Since I was 5 years old until 4 years after college, I called Seattle my home, and technology was intertwined with my childhood. As a kid, I found lots of avenues to my formative years in computing, including access to gopher and telnet via Seattle Community Network, the pre-web BBS scene, and a 5th grade classroom filled with Macs. As a college student, I got to work at various tech startups and ended up at a VC firm after I graduated. There’s not a lot of cities that have the ecosystem to have given me opportunities like that – maybe half a dozen at the most, and Seattle is certainly high up on the list.

Ultimately though, I left after 2006 – it took a lot of soul searching but ultimately one question got me over the edge. Let me explain what that was.

The question that got me to leave Seattle
As I pondered staying or leaving Seattle, I did a lot of thinking about the city from a startup context and what was working and not. Obviously it’s great to have companies like Microsoft, Amazon, Real, and others there – it produced a wonderful tech ecosystem that is thriving and growing every day.

But in late-2006, the social networking world had caught fire, and I wondered:

Post-bubble, when was the last time Seattle produced a world-changing consumer internet company?

And try as I might, I couldn’t shake the idea that while the rest of the tech world in California was producing YouTube, MySpace, Facebook, Google, and others, Seattle had Amazon and sort of stopped.

I wasn’t sure that I would be able to answer WHY, but I packed my bags and figured I’d figure out a theory at some point. A few years later, thinking about the question now, I think it has a lot to do with the kinds of companies being built in Seattle.

Different kinds of companies – Commerce versus Community
My current hypothesis is that Seattle has a strong history in retail and commerce, which has influenced the kinds of companies that are started there. Obviously you have Amazon, but you also have Eddie Bauer, Blue Nile, Nordstrom, Costco, Starbucks, and numerous other online/offline retail businesses there. There are also lots of transaction-focused startups based in real estate (like Redfin) or travel (Expedia).

These retail and transactionally-focused businesses are great money-makers, but because they target in-market buyers for a particular good or service, it means that you’re not really building a huge audience. You end up with the <10% of the general population that is in-market for buying a diamond or plane tickets or a house, not a viral and sticky UGC site you visit every day.

The classic way to build a huge audience is to focus on ad-driven businesses in the world of communication or content publishing, and there just aren’t that many of them in Seattle. (Though congrats to the Ben Huh for marching his horde of cats in this direction – the Cheezburger sites have the #1 traffic slot in Seattle right now) If you look at categories like social networking or YouTube or Twitter, these are more like everyday tools that hundreds of millions of people might use every day to communicate or find the content they want. Those are mass audience driven businesses and end up being high-variance outcomes – you end up with huge hits and also big failures because you need more money-losing years to build up the audience necessary to monetize at the rates you want. (just look at Imeem’s recent firesale even as they had amassed tens of millions of active users)

Different types of expertise – SEO versus viral/social
Similarly, the above influence also drives the skillset involved for one of the key startup goals: Driving traffic. My working hypothesis for Seattle is that it’s a very strong SEO-oriented community, and you have many of the top experts living and working there. The reason, of course, is that retail and transactional sites are mostly found via Google, and it makes sense to develop a skillset around getting that traffic for free rather than paying the search engine for it.

That’s great, but that also closes the door for the all-important knowledge of the viral loop that companies in social gaming are learning now, and what social networks companies learned before them.

For that reason, much of the social gaming and social network action happens down in the Bay Area.

Comments?
In short, years later I think I’ve mostly answered my own question – my hypothesis is that Seattle hasn’t produced mass audience consumer products mainly because it’s focused on down-to-earth charge-users-for-a-product types of businesses that are more transactional than community. I don’t think that’s a good or bad thing – just as you’ll get more biotech in Boston, there’s a specialization in Seattle around commerce/retail. But if you’re doing a social UGC thing, the Bay area is the best place to be.

Seattle folks (or otherwise): Do you agree or disagree with the above? Let me know in the comments – would enjoy hearing your thoughts.

UPDATE: For all the people who think I’m being a Seattle-hater, here’s a similar analysis for the Bay Area: Does Silicon Valley noise detract from long term value creation? It’s a related piece and discusses some of what I’ve noted since being down in SF.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

I’ve been blogging less and less
As many of my readers may have noticed, I’ve been blogging less and less lately – it used to be multiple times a week, then it became once a week, and recently I’ve been blogging once every other week or so. I’m sure I can keep up that pace for quite a while, but it certainly makes for a less interesting blog :-)

Anyway, some of the reasons why I’ve slowed down in my blogging:

Blogging is more fun when you’re meeting people from lots of diverse companies and industries
When I was doing my Entrepreneur-in-Residence gig, I had an excuse to do lots and lots of meetings with people from across the digital media industry. On a single day, I might talk to companies in mobile, ad infrastructure, payments, social networking, games, and more. That was a great opportunity to blog because it’s easy to see connections and talk about ideas across industries. I don’t do this much anymore, so it’s harder to come up with these observations.

Getting deeper and narrower results in boring blog posts
Getting deeper and deeper in an area is a key part of the startup experience – you learn lots of weird things about your particular project, your particular target audience, and your specific industry. This doesn’t translate to great blog posts though, because most of what you learn there is completely inapplicable to other peoples’ situations. Instead, you get articles that are too “inside baseball” and esoteric.

Long blog posts are hard (and get harder over time)
Sometimes I really appreciate Twitter’s 140 character limit because it forces you to be short and sweet. A blog post, in particular my blogs, go the other direction. Over time, this has become a big pain in the ass since I’m not as comfortable posting one or two paragraph blog posts and instead go overboard with essays. I should probably just come up with a word limit and try to keep things down to a more reasonable size instead :-)

News-driven versus writing whatever
Another is getting inspired to write something – it’s a lot easier to write to comment on something in the news, versus just thinking about a particular topic and writing something great there. It’s always helpful to have some inspiration.

Potential changes?
From the above, it seems like a couple experiments might make sense. A big thing I should do is probably to write shorter things, and maybe do more news commentary. We’ll see if that helps at all :-)

Anyway, less excuses – back to blogging!

Followup to Ning’s Viral Loop article

I was recently sent a copy of Viral Loop by Adam Penenberg, which just came out. I was first introduced to Adam in early 2008, when Marc Andreessen wrote us both while Adam was starting to write an article about Ning and their viral loops. That article was ultimately published in April 2008 as Ning’s Infinite Ambition, which you should read if you haven’t. After the article, Adam subsequently spent more time researching the topic, ultimately resulting in the book. I finished it and wanted to share a high-level summary and also talk through some points that the book brings up.

Summary
The book mostly covers a series of case studies from both offline and online companies. These include detailed dissections of viral companies from all stripes, including:

• Offline: Tupperware, Ponzi schemes
• Andreessen’s companies: Mosaic/Netscape, and Ning
• Bubble era companies: Hotmail, eBay, PayPal, HotOrNot
• Web 2.0 startups: Flickr, YouTube, MySpace, Bebo, Tribe, Tagged
• Widgets and apps, etc: Facebook, Slide, RockYou, Zynga

Some of the companies get pages and pages, and others just get a paragraph or two. But there’s a lot of stories that were new even to me, which is always a good sign, since I tend to love reading this kind of stuff.

The book also covers a bunch of high-level concepts about virality, such as the viral coefficient, viral loops, RockYou’s model for calculating virality, etc. All in all, a useful intro to all the major concepts in the field. It’s a great walkthrough of the history of viral companies since the late 90s when some of the formalizations started to happen.

Metrics-focused virality versus not?
One of the interesting distinctions that isn’t made in the book is the trend of startups who use quantitative techniques to optimize their virality versus products that went viral through other means. In particular, a lot of modern techniques are borrowed from the world of leadgen, ecommerce, and advertising, including:

• Formally defining landing pages (and using associated techniques)
• Creation and formal creation of funnels
• A/B testing
• Extensive use of analytics and targeting
• Deep understanding of email marketing, deliverability, and addressbook importing

From my personal experience, it seems like a lot of these ideas about virality ultimately originated from a few small teams here in the Bay Area who have now helped generations of viral companies succeed.

To me, the most important work in metrics-based viral marketing came from these companies below – I’ve listed the companies along with “descendent” startups who took the culture, playbook, and to build the next group of viral companies

• PayPal (Peter Thiel and Max Levchin)
  • PayPal mafia (Slide, Yelp, YouTube, Linkedin, Geni, etc.)
• Jumpstart (Greg Tseng and Johann Schleier-Smith)
  • Crushlink, Tagged, Hi5, others
• Plaxo (Sean Parker)
• Tickle (James Currier)
  • Ooga Labs (Medpedia, Wonderhill)
• BirthdayAlarm (Michael Birch)
  • Bebo, Flixster

There is lots and lots of overlap amongst this group above, and people cross-advise each others’ companies. Let me also caveat that the list isn’t exhaustive, and there are plenty of important VCs, advisors, and entrepreneurs that “get it” and help cross-pollinate between companies. In particular, I’ve found that the PayPal folks are involved in a tremendous number of companies in the Bay Area, and have been teaching their various companies to go viral for quite a while.

That said, I believe that the social relationships above have become less important over time to startups, as the knowledge around designing and optimizing viral loops has become more widespread. Certainly the Facebook economy has taught a wider generation of 20-something developers on how to build highly viral applications, with or without the help of the folks above. I’d note that some of them aren’t as numbers-oriented as copycat-oriented, but it’s still working for many people. As a result, I think the Bay Area is set up nicely to create the next generation of web companies as the bench in this area has gotten very deep indeed.

Who am I missing? Email me or let me know if I am in the comments. Or if someone on the above list would like to graciously identify who taught them the viral playbook, I can help trace the history further :-)

“Viral Loop” stays high-level
One aspect, both good and bad, about the Viral Loop book is that it stays pretty high level. As mentioned above, even after you understand what a viral loop is, you have to understand the tools of the trade well enough to go execute one. Learning the ins-and-outs of direct marketing takes a long time, especially to become an expert.

Adam does a great job keeping the book high-level and relevant to people both inside and outside of the industry, but certainly it doesn’t go into any of the details that have to be mastered to do the actual execution part.

It is for this reason that the total supply of viral experts will always be relatively constrained. Anyone worth their salt would likely be working on an amazing project, early on in the team, rather than working for an established startup. Instead, what tends to happen is that the community operates on a “money + knowledge” type of relationship, in which successful viral experts advise new startups to provide both angel investment and advisory.

The limitations of viral loops as a force multiplier
Another thing that isn’t discussed much in the book, which I think is very important, is the limitations of viral loops. The quantitatively marketed companies that I mention above certainly have their successes, but similarly, many of them are plateauing and failing as well.

The reason is that there are some important factors that are not well-understood by the extended community.

First, I refer to this great presentation by Siqi Chen (of Serious Business) and David King (of Green Patch), called Metrics for Social Games:

The first slide contains a deep truth: Metrics are a force multiplier. If you don’t have a great product, then you won’t get anywhere. But if you have a great product, then it help you build a huge business.

I’ve written about a similar concept in a blog post called Creating value versus optimizing revenue.

Hitting saturation in viral networks
Another important limitation is that there’s a finite number of users out there, and after you churn through all of them, all you have to look forward to is the long plateau. I first wrote about this in my post Facebook Viral Marketing: When and why do apps “jump the shark.”

I wrote that post back in March 2008, and a lot has happened on the Facebook platform since then. This includes an incredible growth rate of the underlying platform itself (now hitting 325 million monthly actives), the appearance of Social Gaming, and it turns out that the current model to beat on Facebook comes from Zynga. They get around the jumping the shark issue by releasing lots and lots of games – 17 on Facebook, 9 on MySpace, 8 on other networks, and 5 on iPhone. And more to come every day :-)

Although it doesn’t seem like much of a problem for most companies to hit the saturation ceiling in the networks they are operating in, it is a huge problem for VC-backed startups because then the story stops being about growth. So for the entrepreneurs who are working on their startups, it becomes important to hit a product/market fit early, and scale then, rather than prematurely going viral without a long-term product direction.

Buy the book here
Hope you enjoyed the post, and you can buy Adam Penenberg’s Viral Loop here.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

omg she’s getting scammed by a duck!

Techcrunch on social gaming scams
As everyone knows, Techcrunch recently published a provocative article called Scamville: The Social Gaming Ecosystem of Hell. Most people will have already read this article, but just to summarize, Arrington argues the following:

• Social gaming companies (particularly Zynga) are making their revenues in a “completely unethical” way
• Users are getting scammed by the offers
• There’s harmful cycle where the scammiest companies earn more revenue, then buy more ads, then scam more people
• Similarly, some users opt in to offers and then cancel, lowering their value, driving out advertisers
• And finally, the industry is in total denial about this

It’s a compelling article, and I would encourage everyone to read it. There’s also another followup article on publishers who decided not to go the offers route, HotOrNot and PlentyOfFish.

Let’s dig in
I am very sympathetic to Arrington’s views, and investigated the issue over a year ago – here’s my blog post from August 2008 on the topic: Super Rewards and the leadgen side of Facebook virtual currency – can it last?

The more I dug into the issue, the more nuanced I decided it really was – things weren’t all bad, actually. In fact, I’ve come to believe that there are plenty of advertisers where this is working for them, plenty of consumers who are happy as well, though these offer guys are leaving a trail of unhappy users.

It’s clear that of all the issues, the user experience must be fixed. And after the user experience is fixed, I think we’ll still be left with a thriving industry, though people may be making less money than they are right now.

I want to drill into some of Techcrunch’s assertions and go one level deeper to look at the evidence.

It does makes the user experience suck
First off, I think everyone is clear that the way offers run right now, they are very confusing for users. If you search for “zynga sucks” on the Facebook.com domain, you get lots of angry complaints, most them about offers. In fact, I did an article a long time ago and got a bunch of random angry comments that clearly had just been searching for SuperRewards, completely unsolicited.

This was a long time ago, so I hope their service has changed a lot – but here’s a sample:

The post is in regards to Super Rewards. From a game user standpoint, I did the offers more when Super Rewards was not managing the offers. Super Rewards is slow to respond to problems from users, and requires proof of the completion of the offer in ways that the offer itself does not require you to do. For instance, to receive points a mini-game was played, many many offers were reviewed, then the game results were given. The offer states that the points would be awarded once the user reached the results page. If the points are not received, you are supposed to file a request for review. Well, Super Rewards would not take as proof of completion all the information from the results page. They even argued with what the results page displayed, even though it was cut and pasted directly from the site complete with the web address. Instead, they wanted two emails, one confirmation email and one confirmation of the confirmation email EVEN THOUGH DOING EMAILS WAS NOT REQUIRED BY THE OFFER.

Here’s more:

I got stung by them 10 days ago. 440 points for a Discover Card application. I applied, and I am holding the card in my hand RIGHT NOW. They say Discover has no record of recieving my info. Really? Well why did they send me a card then? A$holes.

As one of the ripped off customers of Super (assholes) Rewards on Facebook, I have to say that thier service is a total and utter crap to say the least. Of the offers I have spent time filliing in I have only recieved points for 2 out of the 20 or so offers that I have completed.
Any complaints either get a automated response or no response at all.
There are now groups being formed on Facebook complaining about this type of action. I hope the group action gets up and going, these crooks need to be shut down.

Clearly this is not what SuperRewards wants, nor their game publishers, nor Facebook. And like I said, I hope SuperRewards has cleaned up their service since then.

The folks over at Gambit have written a solid article addressing these issues head on, where they discuss 3 game ending user complaints:

1. “I did your offer but didn’t get my points.”
2. “I completed this offer even though it took forever and now I’m getting spammed.”
3. “I completed this free offer and now I’m being charged all this money.”

The article goes on to discuss why resolving these issues is an important part of the game developers job, and how they can’t just say “oh that’s monetization” and not care about it. These user feelings ultimately come back into the game, and create problems long-term.

I would like to see more of the offers companies directly discussing and addressing the user experience problem openly – I think that will ultimately be the positive result of all of this dialog.

Everyone should be in agreement that the offers experience sucks, but no one is willing to do much because making these changes would mean a short-term monetization hit. It’s a Prisoner’s Dilemma where as long as the big offers providers continue in their ways, everyone wants to match them for competitive advantage. Thus my argument that the only player that’s able to get everyone in line would be Facebook.

It does seem to be working for advertisers
Arrington also makes the argument that the offers industry isn’t working for advertisers, and will eventually cause the monetization to crater. After talking to a lot of people on the issue, I just don’t know that it’s true, to my surprise.

Here’s the quote from the Techcrunch article:

And some users aren’t dumb, either. For every user who gets tricked into some fake mobile subscription, there’s another who can beat the system. That’s where the legitimate advertisers, like Netflix and Blockbuster, get hit. Users sign up for a free trial with a credit card, get their game currency, then cancel the membership and start over.

I specifically asked Jay Weintraub to look into this problem earlier in the year, and was genuinely surprised by the results. I figured that it was all a house of cards, but Jay came back to me with the idea that in fact it’s probably working (at least somewhat).

This is definitely required reading for anyone thinking about these issues. Jay did a great job breaking down the issues.

To summarize his analysis:

• The offers ecosystem on Facebook shares some surface similarities with the “Free iPods” incentivized offers industry that ultimately imploded (just read about Adteractive, Gratis, and similar companies for background)
• The volume of leads being produced by Facebook apps is so large that it’s unlikely that the crappy performance is just being hidden in the volume
• However, the pricing on Facebook will likely go down, and companies will make less money in the long run
• The offers may actually be performing, with the working hypothesis being that users actually choose the offer to fill out, versus the “Free iPods” case where they are run through a forced set of offers
• Also, long-term gameplay encourages accountability and repeat purchase

I think all the above points are surprising, and probably right.

Advertisers may reprice, rather than leaving Facebook
Arrington’s also argues that the bad leads will ultimately drive out all the advertisers. He writes:

Netflix has a policy of only paying for a user once. But game developers use a complex set of partner chains to launder these leads and try to get them through for payment. Netflix sees an overall lowering of quality and pays less for leads. Game developers, desperate to monetize, then search for ever more questionable offers to make up the difference. In the end, the decent advertisers are out, and only the worst of the worst remain.

My question is, why they won’t simply be repriced?

If an advertiser is buying leads at $3, but half the users cancel their orders, then why not just reprice down to $1.50? In fact, the best advertisers probably have the best products, and you might argue that their danger of cancelation is actually less than companies selling niche crappy stuff. Similarly, the Facebook leadgen infrastructure is now a big enough animal that advertisers may want to participate just to drive up volume. Even if an advertiser ends up with an additional $10M with no margin, they might do it anyway just to get more heft into their business.

So I agree with Jay’s argument that in the long-run, these leads just all get repriced, and the same set of advertisers (plus or minus) will remain.

Part of my positivity here is my direct experience buying Facebook advertising, which has actually been high-quality and high-conversion, for the most part. I think that the fundamental traffic is good, and thus the offer advertisers can see the same results, if they aren’t obnoxious about it.

It does create value, through product bundling
The other question is whether or not there is actually any underlying value to offers. And as I wrote in a post yesterday, offers theoretically should be good for everyone, the same way that Amazon and Netflix recommendations are good for consumers. The problem has been the execution, due to user experience issues.

Arrington seems to think, however, that getting users to pay more for the offer to subsidize the virtual good is a bad thing. He writes:

[…] Most of these offers are bad for consumers because it confusingly gets them to pay far more for in-game currency than if they just paid cash (there are notable exceptions, but the scammy stuff tends to crowd out the legitimate offers). And it’s also bad for legitimate advertisers.

I think the above statement doesn’t correctly describe how and why offers can add value overall. I won’t repeat the entire post here, except to give the outline of the argument:

• Amazon recommendations is good, and product bundling as a whole is good
• How do you define a “good bundle” versus bad? How will we target offers in the future?
• How does offer + virtual goods bundling actually work?
• Only 1% of people buy at an ecommerce site

If you haven’t read the article, check it out here.

What will happen next?
My working hypothesis is that the following things will happen – and it might take less than a year:

• The offers industry will continue to grow, the # of players will continue multiplying
• This will mean that the competition for doing leads will be cut throat, and no one will think long-term
• Ultimately, Facebook will intervene to preserve the user experience and make users feel safe in the checkout line
• If they decide not to do it themselves, they will heavily regulate the situation
• Otherwise, they will just make their own “clean” version, potentially by building out the Facebook ads into having landing pages, transaction forms, and redirects, rather than just sending clicks

Either way, I predict it will not end well for most of the leadgen players, unless they clean up fast.

If Facebook regulates, I would like to see them do something like this. Think of it as the FDA food packaging guidelines, but instead of calories it’ll talk about total cost to the consumer.

More reading
Here are some of the related articles that I would recommend anyone interested – they are from the view of the monetization gurus, and looking at advertiser-related performance, rather than user experience.

• Will social payment platforms really work long term? (guest by Jay Weintraub, leadgen expert)
• F*** your game offers (by Noah and friends at Gambit)
• Incentive Promotion 2.0 (also by Jay Weintraub)
• The impending doom of Facebook apps (by Niki Scevak, Jupiter Research)
• The economics of app monetization (with Noah Kagan, Gambit)

and two recent posts I just did related to the same Techcrunch article:

• How social gaming offers create value for everyone
• How Facebook could clean up the offers industry

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

If Facebook doesn’t clean up the offers industry, then this guy will

As a quick follow-on of my last post on How social gaming offers create value for everyone, it strikes me that what the industry needs to survive for the long-term is for one of the big players to break out of the stalemate of zero information sharing, and start advocating for sustainability.

Why all the advertising and leadgen companies hide their information
One of the big problems for the advertising and leadgen industries is the massive lack of information sharing between different parties. The reason is that ultimately, there are really just two parties involved:

• The paying customer
• The company providing the end product or service

But then lots and lots of middlemen get involved, including:

• Agencies / SEMs
• Ad networks
• Publishers
• Infrastructure providers
• Data providers
• etc.

Everyone in that extended chain are just middlemen, and their job is that for every $1 of profit, they want to outmaneuver everyone else in the stack to get as much of that dollar as possible. So if an ad campaign is doing really well, the agency doesn’t want to tell the ad network, for fear that the ad network will raise their rates. On the other hand, the ad network can’t figure out which of the publishers in their ad network actually deliver good performance.

This all sucks, and requires a central party to think long-term. That player might ultimately just be Facebook, but could be a publisher like Zynga (though I doubt it).

What information could be worth exposing
In general, I believe the key to thinking long-term on the offers industry would be to expose all sorts of feedback information, out in the open, at a granular level.

Users would also be able to get information like:

• What are they actually signing up for?
• A standardized view of every offer, like a checklist, similar to FDA mandated food packaging guidelines:
  • What is the 12-month cost of this offer?
  • What is the $ value of this offer to the advertiser?
  • Is this a subscription, yes or no?
  • Am I going to get emails?
  • Am I going to get a phone call?
  • Is my information getting shared with any other parties?
  • How can I cancel? (and this should be standardized too)
  • How do other users feel about this offer?
  • What is the cancelation rate?
  • How do I get customer support if I opt in to this offer
• Every offer should link to an “advertiser profile” on Facebook, with comments, ratings, etc.
• Facebook should be able to instantly ban specific advertisers and offers from ever coming up across all of Facebook

For advertisers and everyone else, they would get to see information like:

• Where are my offers showing up? (by app)
• What kinds of users are filling out my leads? (demographics, geo, etc.)
• What is the $ incentive for users? (by app, by $ amount)

Similarly, there is soft information like:

• How are users rating the app?
• How do they feel about the particular offer?
• How often engaged are users? How much churn is in the app?
• How often do they repurchase virtual currency?

For all of the above, I think a lot of companies would hate it in the short run, and a lot of dollars might be banned, but long-term, this would be better for the overall ecosystem.

Let’s hope that something like this happens!

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

The happy meal is the quintessential version of great product bundling

How offers add value
There have been a lot of conversations about the evils of offers in social gaming, and one thing that’s getting lost in the conversation is the potential for offers to actually generate value overall.

Ultimately, offers are about “product bundling” and it adds value to the economy the same way that any product bundling adds value – by giving people more of what they want, often for less. And naturally, some configurations of different bundles are more effective than others, as we’ll see below.

This post will touch on a couple topics:

• Amazon and “relevant” bundling
• How to define good product bundles
• What’s actually happening with offers and bundling
• Solving the 1% ecommerce problem at the Point of Sale

Let’s get started:

Amazon.com and product bundling
When you are shopping at Amazon.com, and you’re in the process of buying a book, and different book is recommended, how do you feel about that? And even more, if you happen to decide you like both books and want to buy them, and Amazon is willing to give you an aggregate discount, how do you feel?

I think that intuitively, the cross-sell and bundling that happens on Amazon is great for the customer experience, and exemplifies the good side of product bundling.

Here’s some additional information about it from Wikipedia:

Product bundling is a marketing strategy that involves offering several products for sale as one combined product. This strategy is very common in the software business (for example: bundle a word processor, a spreadsheet, and a database into a single office suite), in the cable television industry (for example, basic cable in the United States generally offers many channels at one price), and in the fast food industry in which multiple items are combined into a complete meal. A bundle of products is sometimes referred to as a package deal or a compilation or an anthology.

The article goes on to say that the strategy is most successful when:

• there are economies of scale in production,
• there are economies of scope in distribution,
• marginal costs of bundling are low.
• production set-up costs are high,
• customer acquisition costs are high.
• consumers appreciate the resulting simplification of the purchase decision and benefit from the joint performance of the combined product.

Note also there’s a darker cousin to the above, called Product Tying, in which the consumer is forced to buy the whole set and not just one. This can lead to crappier products becoming more successful, and is the kind of thing you can read about in DOJ monopoly cases.

When bundling is helpful
As mentioned in the list form Wikipedia, there are many situations when bundling is helpful to both the consumer and the business. The bundling is extra helpful when:

• The product being bundled “makes sense” to the consumer
  • “Makes sense” often means a complementary good (drink+burger)
  • Or, it might share the same context (2 of product X are better than 1)
  • Clearly targets the same audience (people who like A also like B)
  • etc.
• Also it can be a great bundle if it was something you were going to buy anyway – like if you put two items in your cart, hesitated and took one out, but were then offered the bundle together

Just as in advertising, you need to “target” your bundles and make sure they are as relevant as possible. If the industry continues to deliver irrelevant offers to consumers, then it’s no surprise that ultimately the whole thing will be written off.

I’m sure I am missing many other examples from above – please write in the comments if you have additional thoughts.

Product bundling in the offers and leadgen world
With the above points in mind, you can imagine what is happening behind the scenes in the leadgen/offers world for social gaming.

The product bundle ends up:

• X dollars worth of virtual currency
• Y dollars worth of bundled product (plus Z dollars of built-in marketing expense)

We can look at this from a couple points of view.

For the product seller, if you’re selling a product for $20, and it costs you $5 to make the item, then you have $15 worth of margin to spend on marketing and still break even. Thus as the product creator, you would be excited about buying up to $15 of virtual currency for the user, if it gets them to buy your product. And if you can buy even less currency for them, that generates profit for you and the leadgen networks and publishers between you and the user.

From the user’s perspective, the above deal can work well if the bundled product “makes sense.” If you were already going to buy a Netflix subscription, and you are being offered the same price and you get some virtual currency to your favorite social game, then that’s great.

So when Michael Arrington of Techcrunch writes that it’s bad for users to pay more for in-game currency than if they paid cash, I think that’s just misunderstanding how offers actually work in the aggregate economy:

In short, these games try to get people to pay cash for in game currency so they can level up faster and have a better overall experience. Which is fine. But for users who won’t pay cash, a wide variety of “offers” are available where they can get in-game currency in exchange for lead gen-type offers. Most of these offers are bad for consumers because it confusingly gets them to pay far more for in-game currency than if they just paid cash (there are notable exceptions, but the scammy stuff tends to crowd out the legitimate offers). And it’s also bad for legitimate advertisers.

How offers solve the 1% problem at Point of Sale
Ultimately, the biggest problem that offers solve for advertisers is the 1% problem of e-commerce. That is, at any given time, the number of people “in market” for anything is actually quite small, and the percentage chance that they will actually purchase something is also very small. As a result, if you are at a “Point of Sale” and they have their credit card out, you might as well try to cross-sell and bundle as much related stuff as possible.

The real skill and value created in all of this, of course, is in actually creating useful product bundles rather than the asinine ones I keep seeing. Social gaming and life insurance don’t mix, the same way that Free iPods and life insurance didn’t mix for incentivized leadgen.

This doesn’t mean that offers companies aren’t totally slimy and the industry isn’t broken
I want to make it clear that all of the above isn’t a judgement on whether the offers industry is working or not working. Frankly, it’s probably pretty broken (I’ll leave that discussion for another post). But I do believe that there is some fundamental value being generated, in the long-run, and someone will build a great company around dynamically creating and targeting product bundles at Point of Sale, wherever you are across the internet.

Whoever does figure that out will make a lot of money, and we’ll forget about all of this social gaming stuff.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

Small profitable companies versus VC-backed startups

I recently had an interesting conversation with a friend centered around a key question that’s come up a couple times before:

How transferable are the skills you learn from building a small, profitable company versus doing a VC-backable startup?

This question came up because part of his life plan was that he wanted to do a “real” shoot-the-moon type startup at some point in his career, but before doing that, he wanted to work on a small profitable company so that he could learn more about the process. We had a discussion around the key assumptions around a plan like that, which centered around the question above.

In general, it’s my belief that most of the knowledge isn’t that transferable, and you are better off just trying to do the VC-backable startup from scratch, rather than deferring that experience. In the worst case, if you fail, you still learn a lot about VC-backable startups and what it takes to succeed. Compare this to building a small, profitable company, where even if you succeed or fail, you may not learn what you wanted to learn.

And of course, it’s a perfectly healthy thing to NOT to want to build a VC-backable company, ever. That is a great idea too :-) But for those who want to have that experience but are deferring it, I would encourage you to try sooner, not later.

VC-backable startups have weird constraints
Ultimately, the core of my beliefs stem from the fact that VC-backable startups have to deal with a number of weird constraints:

• they should grow really fast – people sometimes say ideally hitting $50M in revenue in <5 years
• they should be defensible – ideally having real technology that isn’t easily duplicated
• obviously, you want a great, experienced team – ideally experienced operators or cutting edge technologists
• it’s very centered on SF Bay area and less so on a few other areas (Boston/Seattle/NY/SoCal/Austin)
• early stage is focused on proving things out to get each new round of funding, not on profitability (which is a nice to have)
• etc.

Again, most of the above are nice to haves and they are always on some investor checklist somewhere, and are followed loosely/casually in most cases. Similarly, to get in the game, there are significant “community” effects that kick in too – it’s good to have the right angel investors, because they can help connect you with the right VCs. But angel investors are just random people (albeit random successful people), and they sometimes don’t like to give money to strange people from other cities. So they like to invest locally, and only through people they already know.

So the point on all of the above is, VC-backable companies have all sorts of weird constraints on what you have to be able to do.

Understanding these constraints, and working with them, requires a different mindset than if you are just targeting for profitability.

There’s different constraints on Lifestyle companies, aka Small/profitable companies, aka Passive income companies, aka whatever you want to call them
I think most of the constraints above are pretty silly if the only goal is to build a self-sustaining company that can get profitable and kick off passive income. In those cases, you really don’t need all the constraints above, which really take you down a different path.

In those cases, you could really execute your company anywhere – you don’t have to be in the Bay Area. Rapid growth is both unnecessary, and possibly not desired if new users are creating costs! Instead, you might prefer to charge users upfront, so that you can be sure that you can stay cashflow positive. Similarly, it’s fine to just work with your buddies, or family, or whatever you want – there’s less of a need for them to scale the business quickly, nor will their experience level play a role in whether investors fund the company.

What both the two styles of company do share, however, is that you still need to be able to build a product, and build a business for cheap, even if you are going after different goals.

But even with product development, when you are going for a smaller, self-sustaining company, it’s more OK to target niche markets or build high-quality products for slow-growth businesses. You probably don’t want to build for a new market, since that can take a lot of time and capital to get right.

How much do you really learn?
To net this discussion out, my point is that the two styles of companies are different in as many ways as they are similar. Instead of “walk before you run” it’s more like “learn to sail versus learn to bike.” Learning to sail does not increase your chances of success at cycling, and vice versa, as well.

So for all the engineers out there who are thinking about doing small web projects before trying to take over the world – go for the latter :-)

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

My experience in venture capital

As I’ve blogged about before (though quite a while ago), I spent some time at Mohr Davidow Ventures as Entrepreneur-in-Residence – for more about what that job is, read here and here. A couple years before that, I had spent some time at MDV in their Seattle office, towards the end of the dot com bubble, as an analyst/intern. Both experiences were a ton of fun, and I justified the ~3 years in venture capital where I could have been starting companies as an education that would help me later on.

Now, a couple years later, I thought I would reflect a little bit on where the VC experience helped and hurt me relative to actually trying to build a startup. The net of it is that the time was mostly helpful, and a big chunk of knowledge transferred over, but it was mostly high-level stuff. A lot of running a startup involves mastering nitty gritty details, and the VC experience did nothing to help there :-)

For the lazy/impatient, here are some key things I’d say where it can help:

• It helps with traditional investor/entrepreneur information asymmetries
• Lots of tactical holes still exist
• Investors can often oversimplify startup issues, or overmatch on patterns
• Helps with understanding of investor motivations, which can otherwise appear mysterious

Let’s dive into each of these issues below.

It helps with traditional investor/entrepreneur information asymmetries
Some of the stickiest situation for entrepreneurs are cases where they infrequently encounter a situation, which generates information asymmetries where an investor often knows much more. These asymmetries often involve events like fundraising, selling a company, recruiting executives, etc. In the positive case, investors can be helpful and coach startups through these times, which is great. In the negative case, it provides an opportunity for investors to engage take advantage of naive entrepreneurs, which is not so great. This is why sites likes VentureHacks and TheFunded are useful, because they help even the playing field.

Part of the problem for me, however, is that only the General Partners at VC firms end up actually doing deals. All the associates, EIRs, etc often participate, and you see the final deal terms, but rarely get to see all the back-and-forths that end up with the deal getting done. This creates familiarity with the process, but not the battle-tested experience of having gone through lots of nitty gritty negotiations. But even then, you hear about, and know what the levers are, so everything is less mysterious.

(But like I said, VentureHacks and TheFunded are great, and I only wish there were sites with that level of candor about this obscure industry)

Lots of tactical holes still exist
One area where a venture capital background didn’t help at all was dealing with all the tactical details of getting a company off the ground. In particular, the biggest hole by far is hiring and managing people, which gets abstracted at the financial level. Someone in VC-land can talk abstractly about strong teams, but you don’t have to go through the process of interviewing dozens of people to find the right person.

I’ve written up some of my thoughts here on this topic, in a post called “Building the initial team for seed stage startups” where I talk about a couple points I’ve come to believe:

• Hiring T-shaped people versus specialists
• Try to get doers
• More candidate flow solves a lot of problems
• Interview for the actual work you’ll be doing, not skillset trivia
• Raw intelligence is just one factor – don’t overestimate it

There are also some even deeper questions that are unanswered by VC experience, such as how you actually build out a suitable recruiting pipeline? Or how do you interview people where you don’t have the skillset to comment about their competence one way or the other?

I would say hiring is probably one of the most difficult areas to master, and although there are other block and tackle issues – accounting, leasing an office, operations, etc – getting the right people is just a very hard topic. It’s not a surprise that so many startups struggle with it.

Investors can often oversimplify startup issues, or overmatch on patterns
Venture investors often spread their time across a whole number of industries – you look at their websites, and they’ll say they invest in everything from consumer internet to clean tech to life sciences. MDV was no different, and we were responsive to companies across a large number of markets. One VC explained to me early on that you have to respond to what entrepreneurs are producing, and if you get too “top-down” about a particular industry, it gets easy to overinvest in a bunch of mediocre companies rather than trying intently to just focus on finding the best single team and opportunity you can.

Mike Moritz has talked about this before:

Moritz waxed philosophical by comparing venture capital investing to bird spotting. “I rarely think about big themes. The business is like bird spotting. I don’t try to pick out the flock. Each one is different and I try to find an interestingly complected bird in a flock rather than try to make an observation about an entire flock.” For that reason, while other firms may avoid companies because they perceive a certain investment sector as being overplayed or already mature, Moritz said Sequoia is “careful not to redline neighborhoods”.

Continuing with the ornithological analogy, Moritz pointed to Cisco and said, “There’s a lot to be said for investing in the ugly duckling.” When Don Valentine led Sequoia Capital’s investment in Cisco, many others had passed on the husband and wife founding team of Len Bosack and Sandy Lerner.

One of the difficulties for me personally in seeing a wide variety of companies all the time was that it was impossible to not start to pattern match and draw conclusions about the companies that were probably false. You end up in the proverbial “mile wide, inch deep” level of knowledge about that industry, which makes it all too easy to make generalizations. Similarly, there is a drive to simplify your understanding of a company, since you have to socialize it and talk to other venture partners about particular spaces and companies, which also causes oversimplfication.

Contrast this to startup life, where you end up devoting yourself to one company (which may encapsulate many ideas, as you iterate) for the period of years. You end up diving very deep into situations, and learning about all the different details tradeoffs that cause products to be successful versus not.

Helps with understanding of investor motivations, which can otherwise appear mysterious
Finally, one area where having a venture background helped a lot was understanding investor motivations in general. Entrepreneurs ask a lot of great questions, like, “Why don’t investors want to invest in my idea X which will be highly profitable?” or “Why does hot consumer internet startup X lose tons of money but is valued so much?” The answers to these questions drive a lot of investor behavior, which can be mysterious if you don’t know what’s going on.

The interesting part is understanding why VCs are structured the way they do, why they have a 1 in 10 portfolio strategy, and how they think about their Limited Partners. They have a boss too, of course :-)

The major point here is that building medium-sized, profitable companies that aren’t growing quickly is not really part of the venture capital model. Knowing that can help with all sorts of things, such as massaging your business plan into something “sexy” that investors will respond to. Similarly, it will help get everyone aligned on major decisions, such as financing events, exits, exec team building, etc.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

Welcome to part two of a series of articles from Gambit, a microtransactions platform – you can read the first post here. In the last article, we discussed the average revenues earned from various demographics, and this article touches on implications in product strategy. The author, Susan Su (@susanfsu), is a writer, marketer, and Stanford alum who’s currently at Gambit Payments. Please comment with any questions, and enjoy! –Andrew

Ignore Cougars, Follow the Money: 3 social gaming tips for monetizing younger users
by Susan Su, Gambit Payments

Lately, we’ve all heard a lot about the middle-aged housewife. She’s an adult, she’s got disposable income and a couple of credit cards, probably even a PayPal account. In her leisure time, she sits at home and plays social games with her Facebook friends, possibly instead of going out to a movie. She buys virtual goods with real money while you fill your Olympic-sized swimming pool with gold coins.

This is a great bedtime story to fall asleep to, but would you feel so relaxed if you knew you were leaving millions of dollars on the table?

Age is only one factor
In a previous post, we looked at user age as a factor in a game’s overall revenue. We took a bird’s eye view of average revenue per paying user (ARPPU) by age and transaction volume, and saw that older users – the middle-aged housewife (or husband) – brought in ARPPUs that were 2 or 3 times as much as ARPPUs for younger users. Because of low transaction volumes, however, older users represented little more than a tiny speck of total revenues across the developers in the dataset. It became clear that age data – and even ARPPUs – meant little without the context of volume.

On the other end of the ARPPU spectrum, younger users delivered ARPPUs that were fairly unsexy and unvarying, ranging from $2.58 for 16 and 17 year olds to $3.07 for users aged 20 to 29. However, users aged 14 to 29 together make up 91.5% of the total userbase for the virtual goods industry.

$2.58x…millions
A $2.58 ARPPU doesn’t look so bad when you’re selling to millions of users. The massive transaction volumes associated with teens and 20-somethings aligns directly with this group’s share of total revenue across a sample of nearly 2 million virtual goods users. How massive? Over 93%. That is, users in their teens and twenties bring in over 93% of all revenue seen across all games, for all developers in the sample. Even though ARPPUs are consistently modest, transaction volume – and with it, total revenues – are jaw-dropping.

Younger users are cheap, plentiful, and worth your attention.

Given these figures, what’s a developer to do?

For starters, don’t ignore your younger users. It’s true that there will always be transaction and ARPPU variation based on the game you’re building, what you’re interested in, and what resources you have available, but it’s clear that younger users are still the major players across the board. There are hordes of them, and they’re eager to engage in lots of transactions.

Get Them Young: 3 tips to monetize younger users

1. Think volume. Look for the users who are transacting the most, and then make sure you understand exactly who they are (and how they might be changing). For example, today your revenue may be driven by a massive group of teenagers, but what will happen when those teens become 20-somethings? In this series, we explored this question by age, but you’ll also want to think about geography, language, and gender. ‘Think volume’ means:

• Mind your game. If your product is subpar, you shouldn’t expect amazing volumes or revenues, no matter how much you…
• Focus on growing traffic through virality. How can you make your game even more social, more addictive, and more spreadable?
• Get users to complete. Users are 3 times as likely to make additonal payments if they’ve completed at least one offer.

2. Hold on to your users. People of all ages get tired of games easily. The last thing you need is a poor user experience to push users over the edge and straight into the database of a competitor. Do certain offers just rankle your userbase (leading to poor conversions, bountiful complaints, and churn)? While your payments solution’s algorithms will help you find the best offers for your users, there are always going to be a couple that just don’t perform. ‘Hold on to your users’ means:

• Pick out and remove underperforming offers, either individually or by offer category, and address customer complaints. For example, ‘adult’ offers may not work well if your game’s users are primarily 13-17 year olds.
• Diversify your product(s). How can you enrich a single game to be more complex and engaging? How can you offer more complementary games so when a user defects, she defects to another game in your suite?

3. Keep your eye on empty spaces. Yes, Facebook is huge. Yes, Zynga is dominating. But, growth potential is everywhere still. As more users of all ages sign up for their first Facebook accounts, more people pour into the virtual economy. As Facebook grows in locales outside the U.S., so do the games and apps that inhabit its ecosystem. As users get tired of specific games, they’ll start looking for other places to spend their time and money. They’ll probably invite their friends, too. ‘Keep your eye on empty spaces’ means:

• Don’t make a play just because someone else is making bank off of it (for now). Today’s leaders got there because they kept their eyes on empty spaces and filled them, quickly.
• Look for under-monetized user groups. How well is your game doing with young males? Can you work in a way for more of these users to complete their first offer (and open the door to additional payments)?

These should be your main considerations:

Growth
What does the growth trajectory look like for young users? How many of these users are already playing games, and how many more aren’t? The online casual games industry is still young and has plenty of room for growth.

Facebook boasts 300 million active users, with almost a third of these in the U.S. Since the entire population of the United States is just over 300 million, that means approximately half of all U.S. internet users, or a third of the entire U.S. population, are on Facebook.* Facebook counts 70% of users as having ‘engaged with a Platform application,’ meaning that most users have loaded an app of some sort at some point in their Facebook time. Judging by the impressive monthly active uniques the biggest developers are enjoying (51MM for Zynga’s Farmville alone), it seems that games have already taken off on the network. With all this, is there still room to grow?

Yes. Here’s why:

• Facebook has saturated the U.S. market, but that doesn’t mean every Facebook user is playing a game. Yet.
• The U.S. isn’t the only country in the world, either. In terms of Facebook traffic growth rates, the U.S. doesn’t even make it into the top 10. As other economies (real and virtual) catch up, markets around the world should start looking more and more promising for developers looking to monetize.
• People get tired of games. One developer’s churn is another developer’s new user.

As mentioned above, younger users contribute the lion’s share of total revenue for virtual transactions – for now. However, Facebook reports that the 35 and up group is their fastest growing demographic, so will we see this shift reflected in game usage and monetization too? Probably. But until the older users reach critical mass on the network, would you rather be competing hard for the same handful of housewives or slyly going for the many younger users at lower ARPPUs and massively higher transaction volumes?

Changing ARPPUs
Do ARPPUs change as users get older? Will your 15 year old user be worth more after she turns 18, gets a better job, and starts opting for direct payment over offers? We know that the typical 18 year old makes you more money than the typical 15 year old, so from this we might guess that it will pay off to hold onto that user as she ages.

Older users
Should you try to grow your older userbase? As just mentioned, Facebook’s fastest-growing demographic is the 35 and up set. While actively trying to acquire these users (over others) may divert your resources in ways you can’t afford, it’s likely that your game will indirectly absorb the benefits of Facebook’s demographic growth anyway. If everyone else is focusing on winning the middle-aged housewife segment, would you be better off stealthily (and expertly) acquiring the forgotten younger users? Try it. Measure it. Report back.

Conclusions
In parting, don’t buy into a ‘must do’ (eg. housewives) just because it’s popular today. Popularity doesn’t mean it’s wrong, but it does probably mean that lots of other developers are out there thinking the same thing as you. Instead, look at the data and focus your work where the greatest opportunity currently blossoms. Right now, that’s users who are in their teens and mid-20s.

If you’ve been targeting and you’re seeing interesting results, please share in the comments. What’s worked for you, and what would you do if you were a new developer just entering the marketing today?

For specific questions on data or resources, you can contact Susan here or follow her on Twitter at @susanfsu.

Want more articles?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

Designing a viral loop has multiple stages
Viral loops have been featured in mainstream media and there’s even a book coming out on it – but the step-by-step design of creating a new loop remains obscure, and for good reason. I’ve come to believe that creating viral loops is akin to building a software project – at best, it still comes down to a great team, a strong understanding of the tools available, and relentless iteration. There’s no recipe at the heart of it which guarantees a viral process every time, the same way that you can’t guarantee that any software project will result in market success.

There are no silver bullets in viral marketing
In fact, the core of virality ensures that there will never be a dominant “recipe.” If everyone knows how to build a viral loop around social network invites, then everyone will do it, resulting in consumers will become desensitized, which finally leads to lower response rates. Thus this causes the viral loop to unwind, which leads to long-term disaster.

The only way to combat this is to build a viral loop around the core of your product – something that no one will seek to duplicate, unless they are a direct competitor. These viral loops are incredibly effective because they are lasting and sustainable.

I wanted to jot down a couple thoughts on the different stages that viral loop design go through, so that the entreprenurs reading through this can imagine deeply ingrained, user-aligned ways for their products to gain distribution.

Strategize: Stage 1
The first stage of a viral loop is developing the core strategy around the loop. This requires the viral loop designer to think through, step-by-step, how a user will come to find their product and how they will ultimately pass it along to their friends. If you’re lazy, there are lots of recipes to follow from the Facebook ecosystem like quizzes, “find your friends,” and gifts. As discussed above, these opportunities are already becoming less effective every day.

Even if you decide to use an existing recipe, here are some higher-level strategy questions that should be answered before proceeding:

• How does this viral loop fit into your core product?
• What is the fundamental value proposition you are presenting to your users?
• If your loop is successful, will users transition to your core product or will they bounce when reaching the switchover point?

As you might imagine, most of the discussion here is qualitative and there’s very little A/B testing involved.

Implement: Stage 2
The next stage is the rapid development of the core viral loop. This part should hopefully take days or weeks, not months. It will also certainly be wrong. The best advice I can give here is to follow agile development models and to build the smallest number of features and pages to create the initial flow of pages.

As mentioned before, the best implementations are strongly tied to the core product – as a result, if you’re a video site, it’s best if you can somehow involve videos. If you’re a dating site, you probably want to involve dating.

The other implementation advice I’ll give is to treat the viral loop code as an iterative, protoyping process. So copy and paste all you need, keep it in a separate codebase, and make it easy to refactor. You’ll need to do a lot of messy stuff like changing the order of pages or page elements later, and once you develop your own recipe, it’s easy to rewrite it in the “right way.”

Launch: Stage 3
The next step is to beg, borrow, or steal traffic :-) The easiest way is often to pay for it, $50/day or so, just so you have a trickle of traffic coming in.

Optimize: Stage 4
As you get a flow of incoming traffic, this allows you to deeply optimize the experience. This will involve building out some basic infrastructure to do A/B testing, or using Google Web Optimizer, and otherwise. The key thing here, of course, is to measure whether or not the $50/day you’re spending results in traffic above and beyond what you’re paying for – the more the better, and eventually you’ll cross the threshold where traffic scales infinitely.

In this stage, there are a lot of common fixes that you’ll want to consider:

• Shortening the flow of pages (can you shrink a 5 page funnel down to 2?)
• Rearranging UI elements to emphasize next steps
• Testing different value propositions for going through the flow
• Increasing the # of people invited

This optimization stage creates great conflict for product and customer-oriented people. Oftentimes, to get a number to move from 10% to 30%, there’s temptation to do things that users may not be happy with. This might include things like asking for invites multiple times throughout the initial session, presenting an opt-out process for selecting friends, etc. These are all bad and need to be fixed in order to create a long-term sustainable viral loop.

This optimization step can take a very long time (months is not uncommon) as you zero in on the dozens of small and large changes needed to create a viral loop.

After months of work, two outcomes can result:

• You don’t reach your goal, and you’re stuck on traffic
• You reach your goal, and your traffic is going bananas!

If you don’t reach your goal, then it’s time to stop your optimization process. Often the changes that result are just too small to drive substantial increases in metrics. Instead, you’ll have to rework your entire value proposition, which means to either go back to Stage 2 or possibly Stage 1. This means you’ll want to stop A/B testing and start building out a deeper featureset.

Refine: Stage 5
If your optimization step was successful, your work is probably not done. The final step is polishing your viral loop.

This includes figuring out issues like:

• Making your loop as user-aligned as possible
• Building a pleasant user experience and removing unnecessary flows or page elements
• Refactoring the code to move it from prototype to production
• Integrating it into your core product in a way that makes sense

A lot of people are tempted to skip this polish step, but don’t do it! Skipping this step means that your initial product experience will suck, or be offensive.

In fact, when there’s “excess” virality, that’s a great opportunity to make changes to the viral loop that make it nicer or friendlier. In general, if you are getting exponential growth, it’ll be great even if it’s a slower exponential. What’s more important at that point is spendfing your extra growth towards changes that positively impact long-term retention.

On the other hand, if your product is just meant to be short-term mad money, then by all means skip this step :-)

More on viral loops and marketing
For those that are interested, I’ve written more about viral loops and marketing here.

Today we have the first part of a fantastic two part series where Gambit, a microtransactions platform, is sharing exclusive data and analysis for the payments happening on their platform. The author, Susan Su (@susanfsu), is a writer, marketer, and Stanford alum who’s currently at Gambit Payments. She wants startups to make it big, and you to make more money. Enjoy! –Andrew

Age (and ARPPU) ain’t nothing but a number
by Susan Su, Gambit Payments

In the game of life, you’ve heard that age ain’t nothing but a number. In the world of social games and virtual currencies, the same thing goes. The smart developers know to segment by age groups and target towards those with the highest demonstrated ARPPUs. The even smarter developers know that age ain’t nothing but a number – a single, lonely metric that can dangerously limit your view when you exclude crucial supporting data.

In this post, we’re using demographic data from Gambit Payments to get a bird’s eye view of ARPPUs by age and transaction volume. We’ll see that age data – and even ARPPUs – mean little without the context of volume.

A look at highest grossing ages
Which users pay the most to play?

From this data set, you can see that Gambit’s developers got the highest ARPPUs with users aged 50+. In this month, 60 year old users brought in a $7.92 ARPPU, more than double the ARPPU seen with the younger set.

Players in their 40s averaged a $4.39 ARPPU range – a pretty impressive figure still. Going younger, players in the 30 to 39 year old range brought in a slightly lower ARPPU of $4.11 while players in their 20s brought in $3.07 on average. Finally, teenage players brought in ARPPUs in the mid-$2 range.

This data should come as no surprise. Let’s take a look at the main levers feeding into ARPPU:

• Income. How much money does this user or group of users make? In most respects, this lever is straightforward; if the user in question doesn’t pull in an income, they won’t initiate direct payment for your currency. But, that’s what offers are for. Note, however, that offers typically do not bring in the same flashy ARPPUs as direct credit card or PayPal payments.
• Access to which type of payment. What payment methods are available for this user or group of users? Since we’re talking ARPPUs here, a paying user is a paying user – and thus already has access to some type of payment. However, remember that not all payment methods deliver the same dollar value to your pocket.
  • Does this user or group of users have access to credit card payment or PayPal? If so, you’re in luck. These methods typically bring in the highest revenues because they’re relatively easy and impose minimal friction. Note that PayPal penetration may be low in some parts of the country and world, so it’s unlikely that PayPal will be your biggest breadwinner overall.
  • Will they be paying through their mobile provider? With mobile, money travels through lots of different hands – mobile aggregators, mobile operators, mobile payments providers – before it reaches you, a trickle-down process that will affect earnings accordingly. Also, keep in mind that mobile is based on fixed pricepoints, which gives you less flexibility for what people will pay for. Finally, when paying with mobile, there’s also a cap on a transaction’s dollar amount – you can’t, for example, pay for something costing $100 through your mobile service. While mobile payments typically bring in lower ARPPUs, they also have lower access barriers and are relevant to a wider swathe of your users.
  • Will they be completing offers to earn your currency? Offers can bring in decent ARPPUs, but, for certain user groups, may lack the longetivity of direct payment methods. Will your users complete offers, only to decide that they hate the experience and would rather abandon the process – or your community – altogether? How will you deal with this? For further exploration of this topic, see Gambit’s post on user complaints and coping strategies.
• Willingness to buy online. What is this user’s comfort level with online purchasing? If they’re uncomfortable with online purchasing, ARPPUs associated with this user or group of users will dive accordingly. This becomes a particularly interesting question when you start looking at other demographic data in addition to age – you may find, for example, that users in a certain geographic region are more comfortable with online purchasing because of variance in internet penetration or fluency.

If these levers sound familiar to you, you’re doing well so far. Now let’s see how each of these factors works in the context of the data presented above.

Older users
Older users not only have disposable income, they have access to the payment utilities – credit cards, mobile phones, PayPal accounts – that bring their money to your community. Why 60 year olds specifically? You should view the fact that 60 year olds were at the top as a datapoint specific to this set (an outlier) than a generality that should be extrapolated into rule. If you take a look at the groupings, the 50+ group still achieves an average ARPPU (across individual years) of $5.20 – pretty impressive.

At the other end of the spectrum, your community’s youngest paying participants probably don’t have jobs or the disposable income they bring. Their access to direct payment methods is likely to be highly limited or nonexistent. On the other hand, they probably do have access to mobile payments, and can always complete offers. Based on the notes above, you know that payment via mobile and offers will mean lower ARPPUs for these users.

The key here is to know your users – Who are they? How much money do they make? Where do they live? What types of payment methods are available to them, and how willing / able are they to engage with different methods?

Revenue breakdown by age
Finally, does all this mean it’s time to regroup your acquisition efforts and start to go after the 50+ set (or, if you have been already, give yourself a hearty pat on the back and quit working so hard)? Not yet. Let’s take a look at the percentage of total revenue that these groups bring in, respectively.

It turns out, despite impressive ARPPUs, the 50+ group makes a sad showing when we start looking at percentage of total revenue. If we’d halted our analysis at individual ages, or even broader age groupings, and the ARPPUs they demonstrated in this data set, we would have missed the point entirely.

Transactions breakdown by age
For Gambit developers, 50+ was the goose that never laid its golden egg. All the users in this entire group represent only half a percentage point of Gambit developers’ total revenue for this period. This isn’t because there are 200 age groupings, either. Let’s take a closer look.

Wow. The 50+ group represents a meager 0.3% of total transactions – a figure so small that it barely registers a speck on the revenue radar for Gambit’s developers. Users in their 20s, by contrast, produced 22.5% of all transactions. Finally, teen users represented a whopping 73.5% of transactions made across all Gambit developers. 73.5 versus 0.3… suddenly that $7.92 ARPPU doesn’t seem so significant anymore.

What’s good about the user groups bringing in lower ARPPUs, and how do you optimize their experience to impact your revenues? Conversely, is it possible or worthwhile to improve transaction volume for the highest ARPPU groups? In next week’s post, we’ll go over the strategy implications of the data we presented here and contrast a few approaches to make you more money.

For data geeks
If you prefer to look at the above data in a neat table instead of a fancy pie chart, here it is:

Hope you enjoy this data from Gambit Payments, and part 2 of this article will be coming soon!

[Andrew: Thanks again to Susan for putting together this great post!]

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

A quick rant:

Every time you talk to an ad network or leadgen network or whatever, if you ask what their differentiation is they will say “targeting.” That’s probably wrong, and let me tell you why, based on the recent announcement of the Netflix prize winners:

Netflix was able to wring three years of research to nudge its recommendation algorithm up 10.5 percent, at a cost of $1 million in prize money — a stunning feat on its own.

This means if you combine dozens of the best machine learning people in the world, some of the cleanest datasets, you get a measly 10.5% increase. Compare this to starting a new ad network where you end up with noisy datasets, lots of crappy traffic, and a small team looking at the problem – that’s not an easy path to disruptive change. In general, 10% is not a big enough number to counteract the other economic drivers in the ad market, which revolves around better deal terms, a larger selection of advertisers, better ad inventory, etc.

I would guess that you need a number closer to 50% lift or higher in order for an upstart to dramatically change the ad landscape and neutralize the weapons of the mass of ad network players.

I think disruptive change will come not from algorithms, but rather two other areas:

• Better ad inventory: New websites and mechanics emerge all the time, and who knows what happens when you put ads on them? It was clear, until they tried it, that with the right ads search can be >30% clickthrough rates or more, which is unheard of.
• Better data: The other big opportunity is in using specialized data to drive your algorithms – rather than basing everything off of domains, cookies, and ad impressions like everyone else, there may be ways to extend the targeting to unique datasets that no one has access to. This is what’s happening in the world of retargeting.

The Netflix prize also included people adding in additional data, and that’s factored into the 10.5% improvement. Anyway, the point is, increasing performance on stuff like this is very hard, so when an ad network tells you about their targeting, you should push them instead on their revenue split ;-)

If you don’t watch out, after a couple months of A/B testing, your product will end up looking like Las Vegas!

Why A/B testing and visual design come into conflict
It’s great to implement consistent A/B testing in their product process, but then it becomes even harder to keep a consistent visual design while doing test after test. This tension comes from the fact that A/B tests push you towards local maxima, making the particular section of page you’re testing high-performance, but at the expense of the overall experience. As a result, there’s a lot of temptation to “hack in” a new design, the way that software engineers have to “hack in” a feature – but this is short-term at best. This often means adding a bold, colored link to the top of page with “NEW!” or adding yet another tab – these are all band-aid solutions because once you get to the next set of features, it’s not a scalable design to have 100 tabs.

Each of these competing features, taken by itself, moves the needle positively. However, there isn’t a great way to measure the gradual “tragedy of the commons” effect to the overall user experience. Each new loud page element competes with all previous page elements, and must be louder as a result – this leads to the Vegas effect that many Facebook apps end up in.

To really solve this problem, you need a central design vision – there’s no way around that. It also helps a lot to have a flexible design that embraces A/B testing – you can work with your designers to make this happen through modular, open elements.

Closed designs make it hard to add or remove content
Let’s take a particular example and look at it – this might be a standard example on a page like a video or otherwise:

It looks nice, but also has tremendous sensitivity to the content and an inflexible design that makes it hard to test new content. To be more specific, ask yourself the following quesitons:

• If you wanted to add a comments count in addition to views and votes, how would you do that?
• What happens when the views number gets beyond 10,000?
• What if you wanted to add favorites, or flagging for inappropriate content?
• If we decided to hide the thumbs down, how would this visually balance?
• If we wanted to fit more thumbnails onto a browse page, how easy it is to shrink the main thumbnail?
• etc.

The above design is an example of a “closed” design where everything fits just right, but makes it very difficult to add or remove elements. There’s an exact balancing of all the parts of the element, which makes it very sensitive.

Many of the solutions to the questions involve either require building out new pieces next to the element, which throws it off balance. Thus, if the above were used in an A/B test, the visual look would be immediately ruined.

Open designs that are A/B test-friendly
Let’s compare this to the elements below, which have a more modular design that can scale vertically:

The above elements don’t have the same “just right” visual appeal, but make it much easier to add and remove content. The key design decision is to add multiple bands of content which can be grouped together and extended vertically. Ideally, you would never end up with a repeating tile of 4-buttons and 3-stats, but you could certainly test it much more easily than with the closed design.

Here are some of the variations that can easily be tested:

• Switch the title section and the stats/buttons sections
• Add and remove buttons (or no buttons!)
• Add and remove stats (or no stats!)
• Combine price tags with other stats
• Try different buttons
• etc.

Following an open design on page elements enables substantial A/B testing within some flexible constraints. Now you may still be tempted to do something crazy like big hover overlays, <BLINK> tags, and other stuff, but at least you can make it easy to test a wide variety of low-hanging fruit. It also makes the owner of the overall visual design able to maintain a central “style guide” while still offering enough flexibility to keep people creative.

This same idea of open designs with horizontal bands of content can be applied to whole pages too – let’s examine a page from the king of A/B testing, Amazon.com.

Open page layouts
From the snapshot below, you can see that Amazon groups the center column of content – each element has a title explaining how it is, a list of items, and a navigation link to see more. This is also true with the item detail pages, which use a similar grouping to show everything from similar books to reviews to other elements. These pages can get very long, but because most of it is below-the-fold, it’s easy to get away with.

I’ve been told that this modular design enables Amazon to take a “King of the Hill” approach to testing each horizontal band of content against each other. Different software teams will create different kinds of navigation and recommendation, and if it causes people to click through to buy, then it floats up higher in the page. This systematic A/B testing is much more easily enabled when there’s the design flexibility for that sort of thing.

Here’s a snapshot for a reminder of what this looks like:

While you may argue that Amazon’s design is cluttered and actuallysucks, on the other hand, this approach lets them take a very experimental approach to pushing out features. It makes it very low-cost to implement a new recommendations approach and try it out without needing to figure out how to design it into the UX.

What’s next? Modular user flows?
Of course, if you can take a modular approach to scaling individual page elements or entire pages, the next question is whether you can take this approach to user flows.

I’ve never seen anyone do this, but this is how it might work:

• Any linear user flow is identified in a product (like new registration, payment flow, etc)
• This flow might be 1 page, or broken into N pages
• Similarly, every individual page might have a bunch of fields (like photo, about me, etc.)
• As part of the A/B testing process, you might want to drop a new page (or new fields) into the flow
• Then an optimization process shuffles pages throughout the flow to identify the best page sequence and page-by-page configuration

You might imagine something like this could be a very powerful process as it would allow you to identify whether you should offer a coupon pre-transaction or post-transaction, or on any given page, where an input field should be placed.

For those who want to know more, I have written a bunch more about A/B testing here.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

Pretty fascinating slides – it’s 128 pages, but worth flipping through – see below. Those on RSS feeds, you can find a link to the presentation here. I found this via Bob Sutton, who writes:

This slideshow was on a number of blogs over the summer (see here) , but I wanted to make sure that everyone saw it and, frankly, to get a post here so I have a record of it.  Apparently, Reed Hastings, the amazing CEO of Netflix, put-up a set of 128 slides that is a “reference guide to our freedom and responsibility culture.” I realize that most 128 page slide decks are deadly dull, but this is an exception.  You may not agree with all their values and approaches, but on the whole I think you will be fascinated by the detail and thought.  Now, I have no inside knowledge of what it is like to work at Netflix, but if this is accurate, it is a pretty impressive company — frankly far more enlightened than most in SiliconValley.

Low-fidelity prototyping versus high-fidelity prototyping

In my discussions with designers, one of the interesting recurring conversations is the tools and process they use to prototype and mock up experiences. In particular, there’s a lot of divergence on how high or low-fidelity to go with a prototype.

For designers that primarily come from agency backgrounds, I’ve found that there’s an emphasis on quickly getting to a near pixel-perfect mockup, and the variations are minor in detail. In that worldview, the ideal deliverable is a single version of something that feels high-quality and gets minor feedback from clients. In the client-agency model, if you give your clients a bunch of rough mockups that seem low-fidelity, then you risk looking unprofessional. Or worse yet, you might get a ton of diverging comments that you then have to work out and iterate – in some cases that’s the last thing you want to do.

This is especially not a good situation for companies that focus on products delivered to customers –  in that case, you mostly want your product to be the right one, no matter how many iterations it takes. As a result, low-fidelity prototyping can be really useful because it aids an iterative, customer-focused approach rather than one where the Great Designer comes up with something directly from his brain.

Here’s a couple of the main advantages:

• Get better and more honest feedback
• It’s great for A/B testing
• Make the cost of mistakes cheap, not expensive
• Refine the page flow, not the pages
• Figure out the interaction design rather than the visual design

In addition, after I’m done arguing my point, I’ll recommend some of the tools that have been useful for me in doing low-fidelity prototyping.

Get better and more honest feedback
The first time I really undestood the power of low-fidelity prototyping was when I started doing usability tests on consumer products I had built (This was years ago). Initially, I wondered why anyone did paper-prototyping? I immediately concluded that it was due to a deficiency of many designers that they couldn’t write code, and thus couldn’t do HTML mockups of the products they wanted to build.

But once I started getting people to view and interact with my prototypes, I realized that one of the big problems was that people didn’t give good feedback when the prototype you present to them is too perfect. Rather than telling me about the really high-level things, like “does the value proposition make sense?” instead they would focus on colors, fonts, the layout of the page, etc. And furthermore, they didn’t feel that they could really jump in and build on top of the ideas you showed them, because it was far beyond their capability to duplicate.

Compare this to a simple exercise where you are using hand-drawn cards or drawing paper and are literally sketching stuff out during a customer interview – you’re much more likely to try something out, and have the person you’re interviewing grab your pencil and say “no, more like this!” And that’s exactly the kind of interaction you want.

It’s great for A/B testing
As for as a metrics-driven approach goes, you have to remember that techniques like A/B testing fundamentally thrive off of variety. In particular, it thrives off of variety at the UI layer, where many small UI changes that cost very little technically can be tried out and optimized. As a result, you don’t want 2-3 pixel-perfect mockups, you actually want 10 or 20 rough mockups where you can select only the most high-variance candidates.

Some of the highest variance stuff has to do with changing the order in which you do things, or opt-in versus opt-out, or richer AJAXy interactions. These are all things where it’s easier to generate many candidates through low-fidelity prototypes since you’re often looking at things form a systems level.

Make the cost of mistakes cheap, not expensive
One of the hidden benefits of having a low-fidelity prtotyping process is that it makes changing directions much easier, which naturally facilitates a collaborative design discussion. When you’re using a customer-driven product philosophy that incorporates a lot of outside metrics and qualitative feedback, you’ll probably get multiple people involved in the design process. If it’s done by one person or a small group, and is polished significantly before it reaches the greater group, one of the problems is that it discourages collaboration. It’s very hard for people not to get defensive when they’ve spent a lot of time polishing something only for it to get changed significantly. Using a low-cost process makes it so that you can try a lot of variations cheaply, without any of the emotions involved.

Refine the page flow, not the pages
One of the highest leverage design  decisions you can make is not about the look of an individual page, but what happens before and after it. For example, you can take a multi-step process and condense it onto one page, or change the ordering of something so that you do something and then register, rather than the other way around. These kinds of design decisions ultimately focus on the order and flow of the user, rather than the look or interactions of any specific page. If you go with a low-fidelity, then it’s easy to draw lots of small pages and link them up in a flow, and do things like cross pages off, change the ordering of a funnel, and lots of things that feel natural when the prototype is very rough. Otherwise, it’s too easy to get caught in the details on the “right” way something works without exploring the options.

Work your way up from low-fidelity to high-fidelity
Of course, you want to make sure that you use the right process for the right job. So there’s nothing wrong with high-fidelity prototyping, especially when you are in the later stages and thinking about issues like branding, look and feel, and all those other details. One way to keep this process going is to have multiple rough prototyping checkpoints so that design decisions are constantly getting refined – maybe the first step is a sketch on a paper, the next is a rough mockup on the computer, then a detailed mockup, then a rough built-out version, and then iterate to the final product. These steps make it so that all the design decisions are well understood, refined, and debated all the way through.

Tools I recommend
Finally, a couple recommendations on tools for paper prototyping:

• Number 2 Pencil :-)
• Giant art pad for drawings – you can get these at an art shop or office store
• Balsamiq (check out the video on the linked page)
• Macromedia Fireworks

In general, nothing beats pencil and paper, but that’s just me ;-) I’ve been told that for people who aren’t comfortable drawing, using tools like Balsamiq helps quite a bit.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

Seed funding and building out the initial team

One of the most exciting events for a startup is landing seed funding, which transforms a “2 dudes in a living room idea” into something with much more potential. I wanted to summarize a couple things that are relevant for this stage, learned from personal experiences and conversations with other entrepreneurs. This blog is targeted at startups who have raised their first $500k-1M of funding, which often leads to hiring 4-6 people – this first batch of folks is critical, of course.

Here’s a quick outline of some of the things I’ve encountered:

1. Hiring T-shaped people versus specialists
2. Try to get doers
3. More candidate flow solves a lot of problems
4. Interview for the actual work you’ll be doing, not skillset trivia
5. Raw intelligence is just one factor – don’t overestimate it

There are many more topics, of course, but this is a good start – let’s dig in:

Hiring T-shaped people versus specialists
One of the truisms in startupland is that everyone has to wear many hats – backend programmers might have to pitch in and do some feature work, designers might have to write some marketing copy, and the CEO might have to vaccuum the office ;-) Just as importantly, if you believe that startups are fundamentally undergoing a process to learn about their customer and the market, then you need people who are  versatile who can see distant connections between a variety of topics. So you want generalists, but a specific kind.

I’ve come to believe that the first batch of people you want on your team are going to be T-shaped, meaning they are broad in a bunch of different areas and deep in a particular one. The breadth of skills gives them enough common context that they can have conversations with anyone on the team about anything, but the depth gives them a source of knowledge that makes them vital to the team.

Testing for this can be as simple as asking a deeper set of questions when interviewing candidates, and asking them to do exercises that are outside of their stated skillset. Most engineering interviews are specialized enough that there are coding questions, but not many that I’ve seen also include an interview around product creation or UI design. Similarly, when discussing candidates, you’ll want to give equal weight to their depth area as their breadth areas.

Watch out for people who are so deep in one area that they seem to be overspecialized – it can be a signal for a lack of interest for pitching in on areas that might be vital for your team, or they may have nothing to do if your startup inevitably changes directions.

Try to get doers
It’s very important to hire people who are execution-focused early on. You just don’t have a lot of room for senior people or “philosophers” that don’t immediately contribute value in the product development process. When it comes to seniority, I’ve often liked to hire people who have recently had titles as team leads or directors, but nothing more senior. That way, you get people who are used to the responsibility of leading a team, yet are still low enough to the ground to have immediate impact. This is why people who are fresh out of consulting or banking backgrounds make for impractical partners – they are too focused on strategy and financials when you really should be focused 100% on concrete products and customers.

The other type that you find that’s not execution-oriented is the philosopher type. These folks often interview really well, are familiar with a wide spectrum of things and often experiment with new technologies all the time. The hard part about adding these folks to your early team is that they may be more interested in reading blogs and indulging themselves intellectually than really working hard on a team to get a lot of work done.

More candidate flow solves a lot of problems

For most seed stage startups, getting the first 2-3 people usually won’t be a problem – you’ll have people in mind, or people who are in your immediate group of friends who are easily accessible. What’s much harder is once you move beyond your immediate network, where you may find:

• People you want have jobs and aren’t interested
• As an entrepreneur, you know lots of entreprenurs who want to start something, not join something
• Lots of “OK” people who are interested, but who are hard to get jazzed about
• etc.

It’s easy to immediately get into a state where bars get lowered, things you don’t want to get accommodated are, and all sorts of other problems. Or you’ll have interviews where the person was OK but not great, and you really want the skillset.

All of this hand-wringing can be solved if you find a repeatable model for contacting qualified people and getting them in the door. I’ve found that when you start hiring for a new job role, it’s hard to figure out who a perfect candidate is – it isn’t until you see 10 candidates that you start to hone into what you really want and like. Figuring out the repeatable process that works for you is the hard part – but you want to find communities where your ideal candidates are already involved, and start talking to as many of them as possible. This may be Newgrounds for Flash people, or the Firefox extensions directory for browser folks.

Interview for the actual work you’ll be doing, not skillset trivia
I’ve previously written a bunch about my feelings on this topic, so I’ll just link here. The short of it is that I think most interview processes suck because they aren’t actually tests of what it would be like to actually work together. The ideal interview, imho, is just to interview, then work together for 2 months and do a checkpoint to see if it’s working OK. But because most people looking for a job aren’t willing to do that, having a 3-day “working interview” is a reasonable substitute as well.

Raw intelligence is just one factor – don’t overestimate it
All the young, energetic entreprenurs I know want to hire other people like them – high-horsepower people who work hard. As a result, you can organize an entire hiring process around intelligence, full of puzzles and brainteasers, and reward anyone who thinks quickly on their feet. I’ve found that this actually sucks as a minimum bar for hiring people – it’s just as important to evaluate things like passion in the area you’re working in, their reasons and goals for being at your startup, etc. The reason why you need to evaluate this stuff is that startups are really hard, and often take more time than you think they will – as a result, it’s important to understand peoples’ motivations to make sure it’s a good match from the beginning.

A couple of the things that are useful to evaluate:

• How do they see work fitting in their life? Does the style match? (Mornings vs late, hours, etc.)
• Level of process and improvisation – how are decisions made, how well are things spec’d and defined, etc.
• Mutual goals and motivations – money versus serving a goal versus learning versus others
• What stage of startups do they like? Seed, mid, later on? Why?
• Long-term goals – create a great lifestyle company, or try to go big?

I think questions like the above should get equal billing as testing for skillset – these are the kinds of things that impact long-term collaboration and performance as much as mastery of skillsets of knowledge.

Figuring out the alignment of these soft skills reminds me a lot of this great interview with John Doerr of Kleiner Perkins, where he discusses Missionaries versus Mercenaries – you can watch the video here. Ideally, you find people who really understand and believe in the mission of the company – and if you have really smart mercenaries in the team, it may work well for when things are going well, but there’ll be significant issues if there are any hiccups (which there are bound to be some).

As a side note, this is often a problem with hiring metrics-oriented people, because their passion and interests aren’t as much in the value that’s created through the product as much as figuring out how to make the metrics go up. This can create a very revenue-oriented mercenary culture that leads to weird company vision problems. I think the ideal scenario is to find people who have a passion for the particular product you’re bringing to market, and then training them to be metrics-oriented, versus taking people who love numbers/data/algos and trying to train them to love a particular product area.

Comments? Any lessons to share?
If so, please comment below – would love to hear from other early stage startups and their lessons in hiring the first 10 or so people.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

Why does everyone hate on desktop applications? Homer and spiderpig love them.

Desktop apps have better retention, while websites have better user signup rates – which factor wins?
There’s a lot of conventional wisdom that it’s dumb to build a desktop app, and it’s marketing suicide to do so.

This argument usually has two parts:

• Poor conversion rates: Maybe 1-2 out of every 100 – will download your application
• Poor virality: Since desktop apps are often standalone utilities, they lack the social element that makes them viral

You can compare this to web products which often have 10%+ registration rates, up to 50% when they are through friend-to-friend invitations.

It seems obvious that going down the web route is a slam dunk, but in fact it’s not – desktop applications often have a better long-term retention, and this can easily offset the lower download+install rates. The way to look at this is that the number of active registered users is a function of signup rate AND retention, and you can balance one with the other. (And of course, ideally you have both). To me, this discussion opens the way for more innovation in browser extensions, downloadable apps, and other low signup % products as long as the long-term value is great enough.

Note that this blog post will focus exclusively on the signup rate versus retention rate, and leave the virality discussion for another day. Let’s dig into this further.

Comparing applications versus websites
Here’s an example of the simple differences between the two channels:

Starting with the same number of new unique users, it’s obvious that this can lead to a huge difference in account registrations.

However, because web products are so easy to get into, they are also easy to get out of – it’s hard to be sticky. The one true retention mechanic is using e-mail notifications to get the user back. Compare that to a desktop product that use techniques like:

• Open itself whenever a file extension is clicked
• Install itself on the system tray
• Add itself to the desktop
• Start up automatically when the OS loads
• Run nicely in the background to pop up when appropriate
• … and many other retention-happy features

(Of course, you should never use these techniques without contributing value to the user, lest you get uninstalled and reported to Symantec).

Similarly, there is also emerging a world of in-between web-triggered applications like Firefox extensions and Adobe AIR apps, which are easier to install but also take advantage of a wider set of retention hooks to stay relevant.

So when all of this has been taken into account, you can see how our 100,000 new users fare after a couple time periods below. Here’s a table that describes two retention rates period-over-period, and how many active users are left after each period, starting with the initial numbers (1k vs 10k) discussed previously:

As you can see, after the course of 5 months, the application now has more active users than the website, even though it started with a 1/10th the registered users.

Note that retention rates usually improve period-over-period, and are not constant as shown above – I’m just using a constant retention rate so that we can simply the math in the next section!

Looking at the math
For the readers that fall asleep when an equation is shown, please skip this section :-)

Ultimately, the function for describing the number of active users at any period is:

# of active users at time t = initial user signups * (retention rate)^t
= (new users * signup %) * (retention rate) ^ t

So if you have 100,000 new users, a 10% signup rate and 50% retention rate, then your equation looks like:

# of website actives at time t = 100,000 * 10% * (0.50)^t

If you want to calculate when a website’s active users falls below a desktop app’s active users, you can set the two equal to each other and solve for t:

100,000 * 10% * (0.50)^t = 100,000 * 1% * (0.80)^t
10% * 0.5^t = 1% * 0.8^t
10% / 0.1% = 0.8^t / 0.5^t
log 10 = log(0.8^t/0.5^t)
1 = log(0.8^t) – log(0.5^t)
1 = t * log(0.8) – t * log(0.5)
1 = t * (log(0.8)-log(0.5))
1 = t * log(0.8/0.5)
t = 1 / log(0.8/0.5) = 4.9

Thus, after 4.9 periods you’d see the higher retention product start beating the high signup product. In the cases where they never intersect, you’d get a negative number there. I will leave it as an exercise for the reader to solve this in the general case where you know that a website’s signup rate is X times more than desktop app, and Y times in retention.

Conclusions
Ultimately, I believe my calculations show that desktop apps have natural advantages (and disadvantages), but are not strictly worse than building a web property. You still need a long-term value proposition that drives great natural retention. You need expertly-done “hooks” into the OS, email, and other notification systems that encourage repeat usage. And finally, you need social hooks into viral channels (whether web or beyond) that encourage virality and user-to-user interaction

I think it’s not a surprise that there have been great success stories in desktop apps in recent years, such as Skype, Twitter clients, new browsers, and other tools that follow the design patterns of the above. And of course, nothing beats building a killer product that spreads naturally through word-of-mouth – that said, you can stack the deck using great retention and virality :-)

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

Sometimes platforms can be dangerous to your health…

Recent platform news
I’ve been interested in the rush of recent news about the various challenges that app developers are facing relative to the platforms they’re building on, whether that’s Facebook or iPhone:

• Couldery Shouldery: the Lookery post-mortem
• CNet: Does iLike price show the cost of Facebook dependence?
• The Case Against Apple in Five Parts

Reading the excerpts
Each of these stories is slightly different, but worth repeating – here’s the relevant paragraph from Scott Rafer’s Lookery post:

So far so good on using an ephemeral opportunity to create a company, but this is where I place Coulda-Shoulda #1. We exposed ourselves to a huge single point of failure called Facebook. I’ve ranted for years about how bad an idea it is for startups to be mobile-carrier dependent. In retrospect, there is no difference between Verizon Wireless and Facebook in this context. To succeed in that kind of environment requires any number of resources. One of them is clearly significant outside financing, which we’d explicitly chosen to do without. We could have and should have used the proceeds of the convertible note to get out from under Facebook’s thumb rather to invest further in the Facebook Platform.

Similarly, here’s the excerpt from the recent article about iLike:

Some in Silicon Valley have speculated that MySpace isn’t willing to pay more for iLike because it fears Facebook will boot iLike once its main rival takes control of the service. But that doesn’t go far enough in describing the situation, said one of the sources. What has pushed iLike’s valuation down is a problem with control. The company’s managers have no way to prove to potential acquirers that their business model has a bright future because they can’t predict from one day to the next which direction Facebook’s Platform will go. The source said that leaders at iLike, or any other company on the platform, are not truly in control of their fate–Facebook’s Mark Zuckerberg is.

“The cash flow of any company doing business on Facebook’s API, or Facebook Connect, or Facebook platform is inherently at risk,” said the source. “The multiple that an investor can place on that cash flow is not that much greater than 1, because you never know at which point Facebook could change the terms of the relationship or change the technology and cut off that cash flow.”

And finally, the discussion on the iPhone platform, which Jason Calacanis makes in 5 parts with the last 3 points involving their App store platform:

1. Destroying MP3 player innovation through anti-competitive practices
2. Monopolistic practices in telecommunications
3. Draconian App Store policies that are, frankly, insulting
4. Being a horrible hypocrite by banning other browsers on the iPhone
5. Blocking the Google Voice Application on the iPhone

The mismatch of agendas
Ultimately, the vast majority of these disagreements between platforms and applications seem to be over the inherent mismatch of agendas between the two parties. Applications seek to maximize their distribution and gain customer share, while minimizing their dependence to the particular channel. For platforms, they seek to control the applications which depend on them, and prioritize the long-term success of the application ecosystem rather than any individual application’s. The ecosystem around a platform is complex because there’s a 2-sided market built in – the customers they serve, and the applications that want to use them. Prioritizing application developers above all else leads to sloppy, disjointed experiences – that’s one of the things you have to admire about Apple’s tight-fisted approach to App Store discovery, payments, etc. I haven’t had a bad experience yet, versus the constant complaint comments I get from disgruntled users whenever I wrote about Super Rewards or Offerpal.

As I wrote about in my previous blog post Benefit-Driven Metrics,  ultimately the platforms should try to help the the applications that build on them make as much sustainable revenue as possible. As long as there’s a long-term business there, more developers will continue to be attracted to building more functionality and richness. I believe that the Facebook economy has (surprisingly) shown itself to be capable to support several VC-backed companies making 10s of millions in revenue, whereas the same cannot be said for the iPhone platform yet. I’m sure someone in the mobile world will figure it out eventually, though.

For any application developer though, the core lesson is – don’t get too comfortable ;-)

Conclusion
How will these recent issues steer the platform agenda in the future? In particular, let’s look at iLike’s exit – what are the implications for other startups?

Will people conclude negatively about:

• music startups
• ad-based app companies
• startups that are building horizontal apps on Facebook/Twitter
• any startups building on one of these platforms

Perhaps it will be all 4, or perhaps just localized to a particular sector. Only time will tell.

Hope you enjoyed this article, and leave me any comments if you have extended thoughts!

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

To my first 10,000: thank you for reading! (And I will try to blog more)

Lately I have been tweeting more than anything else, and if you haven’t followed me already, you can do so here: @andrewchen.

Best,
Andrew

PS. I updated my list of essays recently, so all the latest stuff is on there. I wanted to include it below, for your convenience.

Viral marketing and user acquisition

For web entrepreneurs, growing your userbase is a key challenge, alongside product development and financing. These posts emphasize a quantitative approach to getting traction and growing users.

• What’s your viral loop? Understanding the engine of adoption
• Adwords is not enough for success on the consumer web
• Are your SEO efforts working, or failing?
• Bridging your traffic engine with your revenue engine
• Facebook viral marketing: When and why do apps “jump the shark?”
• How to calculate cost-per-acquisition for startups relying on freemium, subscription, or virtual items biz
• Is your site really viral? Viral Branding versus Viral Action
• Social network marketing: Getting from zero to critical mass
• Viral marketing is not a marketing strategy
• 10 obvious strategies to ruthlessly acquire users
• Growing renewable audiences (a talk at O’Reilly Alphatech Ventures)
• Go-to-market strategies for vertical social products

Engagement and product design

Using principles from game design and analysis of consumer behavior, these essays cover the process of creating experiences your customers will love.

• 10 signs you’re a product fanatic
• 25 reasons users STOP using your product: An analysis of customer lifecycle
• Are people like lab rats? Using reward schedules to drive engagement
• Do you ever say, “MySpace is sooo ugly?” This blog’s for you…
• Does Facebook reflect your true friendships? How about e-mail?
• Facebook Apps: Why they’re focused on fun instead of utility
• Friends versus Followers: Twitter’s elegant design for grouping contacts
• Is your website a leaky bucket? 4 scenarios for user retention
• Public and private spaces, and why YouTube comments are so awful
• Social gaming design – Bartle types versus Web 2.0 participation pyramid
• Social network death spiral: How Metcalfe’s Law can work against you
• Social design explosion: Polls, quizzes, reviews, forums, oh my!
• Talk to your target customer in 4 easy steps
• Technology always changes, but people always stay the same
• The design of social spaces
• User retention: Why depending on notification-driven retention sucks
• Users, customers, or audience – what do you call the people that visit your site?
• Why your friends list gets polluted over time
• Why you should make it easy for users to quit your product
• Your site will succeed or fail in the first 10 seconds

Freemium and online ad monetization

Social web product have unique characteristics as it applies to online advertising and direct monetization levels. These posts cover some of the issues around key topics such as ARPUs, conversion funnels, CPM rates, behavioral data, revenue modeling, etc.

• 3 key ideas from a recent Freemium dinner conversation
• 5 factors that determine your advertising CPM rates
• 5 things that make your social network monetize like crap
• 7 ideas for billion dollar startups in online advertising
• App monetization: Gambit launches, funnel metrics, and ARPU versus “CPM”
• Data portability: Is the social network data you’re hoarding treasure or trash?
• Counting your big pile of Benjamins: 5 startup tips for maximizing ad revenue
• Creating value versus optimizing revenue
• Free to Freemium: 5 lessons learned from YouSendIt.com
• Freemium case study: AdultFriendFinder ARPU, churn, and conversion rates
• How to create a profitable Freemium startup (spreadsheet model included!)
• How NOT to calculate ad revenue
• Online advertising during a recession: 5 key trends for ad-based startups
• Vertical ad networks: What are they, and why did Cox just buy Adify for $300MM?
• Virtual goods summit video, The Whirled Case Study: metrics for the virtual goods business
• What’s the value of a user on your site? Why it’s hard to calculate lifetime value for social network audiences
• Your ad-supported Web 2.0 site is actually a B2B enterprise in disguise
• “Stealing MySpace” and my personal experience monetizing MySpace ads
• Super Rewards and the leadgen side of Facebook virtual currency – can it last?
• Remnant ads and the advertisers who love them
• Ad targeting talk from Community Next: People Not Pages (updated x2)
• Revenue, ARPU, Funnels, and RPM: My talk from Startonomics on Revenue metrics
• Ad-based versus direct monetization: Which one is better for you?
• What would Facebook look like if it sold out to ads? Click here to see…
• Will social payment platforms really work long-term? (Guest post by Jay Weintraub)

Metrics

Without metrics, web entrepreneurs are just flying blind. These essays cover some of the organization and development issues around instituting a metrics system – what to measure, in what order, and how to implement them.

• 5 warning signs: Does A/B testing ead to crappy products?
• 5 steps towards building a metrics-driven business
• Benefit-Driven Metrics: Measure the lives you save, not the life preservers you sell
• The first 6 steps to homegrowing basic startup analytics
• Are you misusing Alexa numbers? (Probably)
• Lessons from the casino industry on engagement metrics and lifetime value
• Obama and McCain: How political marketing has evolved from offline to online
• omg I’m just a startup, I can’t do those fancy analytics!
• Recency Frequency and Monetization (RFM): Optimizing your notifications strategy
• How to measure if users love your product using cohorts and revisit rates
• How to generate awesome test candidates for A/B testing
• Why metrics-driven startups overlook brand value

Media and games

Traditional media, including TV, music, games, and movies are at a crossroads. Here are some thoughts on how the industry is changing and evolving.

• From analog dollars to digital pennies: The crisis in traditional media
• Game design tutorial at the GDC
• MySpace versus Facebook: Analysis of both traffic and ad revenue, using Google Trends
• What every Web 2.0 entrepreneur should know about virtual goods
• Social Gaming Summit: Recap and observations
• Early adopters vs the Mainstream: Google Insights points out websites only used by Silicon Valley nerds
• YouTube vs Webkinz: Case studies for new product adoption
• Prosper.com and peer-to-peer lending in the economic downturn
• 4 major cultural differences between Games people and Web people

Entrepreneurship and startup life in San Francisco

Just a couple thoughts on things I’ve encountered while arriving in SF.

• 10 tips for meeting people at industry events
• 2009 conference schedule for the digital media industry
• 5 ways to break past the San Francisco echo-chamber
• Are Web 2.0 startups wasting their time with Web 2.0 early adopters?
• Bay Area investors I’m following on Twitter
• Built to Fail: How companies like Google, IDEO, and 37Signals build failure-tolerant systems for anything!
• Couple quotes on Facebook in Wired, Fortune, and NYT
• Does Silicon Valley noise detract from long-term value creation?
• How do you find a badass co-founder, Part 2
• How do you find a badass co-founder?
• How do you do concrete interviews for non-technical people?
• How do start a professional blog: 10 tips for new bloggers
• Is blogging worth it? What’s the ROI?
• Moving to SF and joining the tech community – Lessons from my first year
• What’s an Entrepreneur-in-Residence?
• What is your W2SAT* score? (*Web 2.0 Startup Aptitude Test)
• Which startup’s collapse will end the Web 2.0 era?

Weird, contrarian business ideas

One of the best books I’ve read in the recent past is Hard Facts, Dangerous Half-Truths, and Total Nonsense by Stanford Professor Bob Sutton. (He now also has a great book called The No Asshole Rule, which you may have heard of also) In the Hard Facts book, he talks about a variety of different common business topics, and compares the academic research on each of the topics versus what paid management consultants often preach.

In particular, one of those question is – there’s a ton of anecdotes around the idea of The War for Talent, popularized with such phrases like “A-Players hire A-Players, B-Players hire C- and D-Players” etc. Embedded within many of these notions is, of course, the really big assumption that you can actually interview for talent, and that interview processes actually work. In the Hard Facts book, Professor Sutton actually points at a bunch of research that says that in fact, there’s tons of evidence that the hiring process doesn’t work well. And if you look at the marks that people get coming out of a hiring process versus the on-the-job marks they get in their first year in a job, they are actually not correlated at all.

I personally find the idea that interviews being poor predictor of job performance both unsurprising, but also troubling! Interviews predicting job performance seems like one of the core building blocks of American business.

This has been a particularly interesting topic for me to think about because of the differential that exists between technical and non-technical interviews also. All of the non-technical interviews I’ve ever involved in have been terrible, and I’m still not 100% satisfied by my thoughts on how to improve them.

Anyway, I wanted to embed a video interview of Professor Sutton discussing his book below, which you can watch at your convenience. Unfortunately he doesn’t mention job interviews in it, but he talks about a bunch of other interesting stuff.

(scroll down past the video to continue reading the blog post)

Short-term activity used to predict long-term activity
In fact, the core of job interviews really is about using some short-term activity (like dating, interviews, etc.) to try and predict some longer-term success (marriages, job performance). These little prediction scenarios pop up all over the place, and they’re inherently subjective.

Here are some other places where this takes place:

• Investors evaluating a pitch, in order to invest in a company
• Looking at headshots to cast someone in a play
• Reading the script of a movie to greenlight the film
• Being a great premed student versus becoming a doctor
• etc.

Some of these evaluation processes are closely aligned with the actual long-term activity, but sometimes they are not. For job interviews, it would seem that it may not be the same skills to get your resume noticed by a recruiter, then filtered up to a hiring manager, then passing an interview – that is hugely different than actually doing the job in a team setting. This discussion also reminds me of a common discussion I had with pre-meds back in college, where the primary shor-term selection criteria seemed like acing the easiest classes possible, pulling all-nighters, and memorizing obscure biology/chemistry textbooks. However, in the long run, being a good doctor was as much about dealing with people – be it other doctors, nurses, and patients – as it is about having good grades.

Is dating a good way to predict marriages?
OK, before we jump into job interviews, let’s talk about something more fun: Dating ;-)

The traditional notions of dating are a funny thing, because of how contrived it is in many ways. In particular, you might consider dating to have characteristics like:

• Pre-defined activities designed to be fun and happy
• Strong cultural, familial, and peer pressure that specifies standards and traditions
• Low sample size relative to long-term marriage commitment (dating for 6-12 months and then intending to be married for 60+ years)
• Relatively strong separation of stuff like finances, scheduling logistics, etc.
• And of course, no sense of what raising children is like

Contrast this to actually being married, as imagined by an unmarried guy like myself:

• Long-stretches of normal, domestic life that can be exciting, but is often not
• A very long-term outlook on the relationship, spanning 60+ years
• Lots of intermingling of legal issues and logistics
• And of course, the entire process of raising kids, having a house together, is almost an enterprise in itself regardless of the romanic situation

The fact that the before and after is so different, in many ways, means that you better hope that the stuff you learn about the other person while dating gives you a strong view of how the long-term relationship would work.

Job interviews as predicting long-term work relationships
Of course, job interviews are very much like dating as well. Inside of the job interview process, there are a bunch of inherent assumptions about what kinds of candidates are good candidates.

For example, you often have processes that prioritize top schools, or that prioritize “culture fit” and other intangibles. Interview processes often test very specific skills against a “snapshot” of a candidates skills at any given moment of time. Is it fair to ask engineers questions about SQL or specific language trivia when it’s something they might learn or pick up in hours or days, not months? It’s not clear.

The worse issue is around the inherent bias that comes into play. People like to hire people like them, and every startup full of Stanford-educated 20-something guys ends up hiring more Stanford young dudes. Sutton refers to this in his book as homosocial reproduction. How important is it, really, what school you went to? Is it more important your grades? Do you really need to know obscure things about a programming language, or go lower-level, when your day-to-day job is unlikely to utilize that knowledge? I think a lot of these biases come from the people who design the interview, and don’t objectively evaluate success or failure in professional settings.

Do interviews miss the “intangibles?”
Most damning, of course, is if interviews simply don’t test the majority of a job applicant’s fit to a role. Let’s have a thought experiment where for any job candidate, that skillset accounts for 20% of their performance, and other things like motivation, communication, and possibly weird, obscure skillsets actually contribute 80%? Then you can test the 20% all you want, but in any sort of 1:1, contrived conference room interview setting, you can’t scratch the 80%. In fact, you might find candidates that fail most of the 20% but are such amazing fits elsewhere that it’s in fact an awesome match!

Over time, I’ve come to believe that interviews likely test a small amount of a job candidates skills, and you have to more directly test them in realistic work scenarios to get at the other stuff.

How would you more directly test job performance?
In many ways, thinking about job interviews as inherently bad predictors has a strong tie to the Built to Fail blog post I did a while back – except rather than assuming that your code is bad, and needing unit testing to support it, instead you assume job interviews are bad, and you need a larger framework to support that.

So taking the ideas from that post, I would recommend the following:

• Accept that traditional job interviews suck, and you can’t learn much about a person in 30 minutes to an hour
• You should interview MORE people, and potentially lots of weird people that don’t seem to be good matches right upfront
• You must streamline your interview process to handle more people, in larger batches simultaneously – thus 8 hours of 1:1 interviews probably doesn’t scale
• And you should test your job candidates in realistic work scenarios – assigning real tasks to groups of candidates (potentially mixed in with employees), working together in tandem
• Also perhaps instead of focusing your hiring on specific individuals, instead you make offers conditional to entire teams that seem to work well together, and keep them together in their actual job

In many ways, I think this is closest to the Boiler Room or Bootcamp view of the world, which was pointed out to me by a long-term mentor, Bill Gossman. You bring in more people, test them in real-world settings, and hire whoever comes out on the other side, regardless of background or performance. To me, this has the benefit of a truly meritocratic society, where people are hired because of their real performance, rather than what the designers of the interview decided were subjectively important or unimportant.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

Encountering the Silicon Valley echochamber

I grew up in Seattle and worked there for many years, and finally moved to the Bay Area in 2007 because I wanted to be at the epicenter of the startup scene. When I was in Seattle, I always lamented the fact that there wasn’t a “scene” the way that the Bay Area has one. I used to talk about it as one of the big negatives of the region, since you didn’t have the density of events, bloggers, and general activity in Seattle as in the Bay.

Now that I’ve been here for a few years, it’s clear to me that the Silicon Valley echochamber has its clear negatives as well. Being out of touch with the average American consumer is one obvious negative. Chasing down technological rabbit holes is another.

But I believe one especially strong issue is the constant peer reinforcement from fellow entrepreneurs to be working on stuff that everyone deems “hot” or easily relatable. And how easy it feels to be left behind when there’s a “hot new trend” in a particular direction, even when there’s obviously many good markets to be explored at any given time. I would argue that this strong peer reinforcement from fellow entrepreneurs makes it easy to focus on very short-term successes, and ignore long-term contrarian bets.

Peer reinforcement from fellow entrepreneurs
One of the most common conversations you’ll overhear at any startup event is one entrepreneur giving another entrepreneur their elevator pitch. Or, you’ll overhear an entrepreneur giving their pitch to a prospective startup engineer. In fact, I would argue that many startups spend more time talking to other people “in the know,” than they do potential customers, whether those startup savvy people are investors, job candidates, fellow entrepreneurs, advisors.

And just as similarly, a common conversation you’ll overhear is the equivalent of the “hot tip” on a stock – but instead, the conversation will be about a particular market or company. Oh, did you hear that company so-and-so is doing X million in revenue? Oh, the Y space is blowing up.

All of these conversations, as insular, belly-gazing discussions establishing the social pecking order at any given time, provide quick feedback about the entire startup ecosystem. It’s what guides the decisions of many employees or investors to dive deeply into the “hot” markets. In many ways, this is one of the deep strengths of Silicon Valley, that the information is so efficient, and new markets can be quickly identified and exploited by dozens of companies simultaneously.

Copy cats galore
At the same time, these conversations can easily reinforce the feeling for many entrepreneurs that “You’re missing out!” It causes many people to look at the sectors that seem to be immediately doing well, and jump into them as copycats, because it looks like easy money. There’s nothing like the feel of a gold rush to make everyone go nuts.

Interestingly enough, many of these copycats are only in it to exploit the short-term advantages of the market, and some are self-admittedly not passionate about the area they go into. In particular with this economy, a long-time mentor described it as “Silicon Valley’s version of quitting your idealistic startup and going back to work at Microsoft” – meaning potentially soulless activities that generate revenue, regardless of actual passion or long-term belief in the projects as viable businesses.

Focusing on the long-term
As an entrepreneur, I can’t help but look at the short-term choices that get made in an environment like this without some degree of disappointment. There are many brilliant people who could be trying to make the world for the better and really create long-term value, but instead they are engaged in a zero-sum game to extract as much value as possible from the world. Now perhaps as a market, these startups will collectively make the world a better place – such is the wonder of markets – but at the same time, it disturbs my sense of idealism about entrepreneurship.

More importantly, building a startup takes years no matter what the economic environment – maybe 5, maybe more. And if you’re going to be stuck doing something for 5 years or more, then you might as well pick something you’re really excited about. Taking a long-term view, I think, means accepting that many of these new markets will significantly change over time, possibly merge with other markets, or possibly turn out to be too small.

It’s worth thinking about what kind of company you want to be in 5 or more years, rather than just grabbing onto whatever trend seems to be floating by at the moment.

Staying focused on the long-term
So how do you stay focused on the long-term, when there’s so much noise? Here are a couple thoughts for you:

Stop reading blogs so damn much
Every once in a while, I’m busy enough that I don’t read any blogs for a week or so at a time, and you know what? The world doesn’t end ;-) Obviously it’s useful to keep up with how the rest of the tech industry is moving, and where the markets are developing, but clearly there’s a diminishing returns to the minutiae around the startup word.

Have a strong vision that’s flexible yet specific
Another issue is how easily small companies are swayed when the vision is not clearly defined and understood. It’s easy, when internal values and vision haven’t been set, to follow the customer to wherever they would like to go. Or to fast-follow whatever is the darling startup at the moment, or to be swayed by competitor moves. So you need something that’s specific enough to figure out how much external data to incorporate, but also be flexible enough that if you hit contrary data, your entire startup’s core thesis doesn’t fall apart. The tension of the two is what makes this a challenge!

Ignore the competition
For most startups, the market is not clearly defined enough to also have clearly defined competition. In most cases, you’re better off focusing on your customer and learning from them both quantitatively and qualitatively, rather than emulating what your competitors are doing. And in particular, if you are extracting a ton of interesting knowledge about your customer, you may end up with a unique set of insights that would beat whatever you’d get from copying anyway.

Don’t go to startup events
Another common environment where you’re compelled to pitch your startup over and over is startup events. Skip these, and you’ll find yourself thinking more independently from other entrepreneurs.

Forgo short-term opportunities if they are clearly short-term
One very difficult challenge is that along the road to success, there will be many tempting rabbit holes to go down. Many ideas are hard to scale into larger businesses, but make a ton of sense at a smaller scale. Many ideas are also unsustainable, as a hole closes in the market, or because customers don’t get enough long-term value. Of course, sorting out long-term from short-term is the difficult part here.

Move down to the Peninsula, not the city
One of the small geographical differences that exist between the city and the peninsula is that there are far more media-oriented, hipster entrepreneurial engineers in San Francisco compared to Palo Alto, Mountain View, etc. As a result, the city is a fun place to get your company started, as there’s ample idea exchange between all your fellow entrepreneurs. On the other hand, once you get going with your startup, the sheer number of parties, get-togethers, and coffee meetings can get overwhelming.

Be skeptical of opportunities that are both hot, and easy
The most interesting opportunities I hear are ones that appear to be easy revenue, and I hear about them from multiple sources. Many of these opportunities are the equivalent of windows of arbitrage that appear in the stock market – they’ll quickly be closed, and never appear again.

Remember that you only need one big success
The final point I’ll make is that at least as far as startups go, you really only need to find one awesome line of attack on a market, and that’s it. Maybe that takes a month to find, or maybe it takes years. But ultimately, if you are making forward progress on your business and you reach a huge market eventually, it doesn’t matter much what happens between now and then. In this way, having a great deal of patience is very useful if you can systematically discover high-quality, long-term opportunities. This may be harder than the short-term stuff, but it also creates the ability to become a category-defining company.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

UPDATE: Thanks to Marc for reminding me about startup events and peninsula living also!

Why do social products tends towards clutter?
One of the toughest design problems for people working on social products is the inevitable path towards cluttered interfaces and diluted brands, as you try to build more social activities and richness within your product. As a result, these products tend to drift towards “portals,” “hubs,” or “platforms,” rather than clean, single-purpose destinations. This might be good or bad, depending on your viewpoint, but it certainly introduces a number of design challenges when every central “entity” on your site (be it a video page, or profile) has dozens of jumping off points for more complex interactions. Or when you have a “tab explosion” as you bolt on common social application paradigms like blogs, chat, or whatever.

For MySpace, this manifested itself as a massive top menu detailing all the different ways to interact with the site, including Classifieds, Music, Games, Video, Forums, and others. For Facebook, they had to build the Windows-like application bar that shows up on every page and allows access to chat and commonly used applications. It seems as though this clutter is almost inevitable as you try to centralize a wide variety of social activities.

Users push you towards more social activities, not less
The central driver, I believe, for this social activity explosion is that people want to have LOTs of different ways to interact with their friends. These different activities let them have very nuanced interactions that have deep and meaningful social signaling.

Let’s take an offline activity, for example, an invitation to a date – there are lots of nuances that can be read into asking someone to:

• have a quick mid-day coffee
• come over and have a nice dinner
• go to a movie
• have a drink mid-week versus Friday/Saturday
• go out with a group of friends to a music show
• having brunch with your parents
• etc.

All of the above activities provide different social signals based on how big of a time commitment it is, who’s involved, what time of week it happens, how expensive it is, etc. And if you were to build online equivalents of these types of activities, it would be better at each step, since it allows for richer interactions.

As a result of this, your users will always like any new social interactions you push out, and will often suggest/demand new activities.

The social web laundry list
As a result of the demand for new social activities, you inevitably get a series of bolt-on design patterns that recur across many different social products. An incomplete list might include:

• polls
• quizzes
• reviews
• comments
• forums
• chat
• blogs
• videos, photos, and other multimedia
• avatars
• leaderboards
• private/public messaging
• status messages
• etc.

What else am I missing? Suggest some other ones in comments, and I’ll be happy to continue extending this list :-)

Either way, there’s probably some rule that if a new social product these days decides that, “hey! What our product needs is polls!” then the design philosophy of the product probably should be reevaluated. It’s a powerful indicator that the product roadmap is overly focused on short-term user engagement versus a long-term market position.

Drawing the line between “core” versus other
These mechanics are so easily bolt-on-able that it destroys the differentiated value of a product – this happens through clutter, confusion, and overduplication of features relative to other sites. It becomes a trap that weakens the brand long-term, while producing higher engagement in the short-term – quite the devilish dilemma.

Ultimately, to avoid this fate, every product needs to draw a line in the sand on what is core, and what are extraneous social activities that should happen off the site. Or, if not off the site, in a carefully cordoned-off area. Either way, these choices need to get made, otherwise clutter ensues.

Potential solutions
Several companies have dealt with this design problems in different ways – let’s go through all of them:

Solution 1: Build everything
In the MySpace example, the site ultimately decided to incorporate a very large chunk of all the functionality they could think of. Just explore the top menu bar, and I think you’d be surprised by how much product is sitting inside of there.

Solution 2: Open up the CSS/HTML layer
Interestingly enough, MySpace also used another method of allowing users to extend their profiles by allowing people to just copy and paste arbitrary CSS/HTML. Another company that did this is eBay, as well as many blogging sites. Outside of the obvious security issues, the nice part about this is that this is a really simple integration that works with many different kinds of tools and widgets.

Solution 3: Provide a rich onsite platform
This is the Facebook/OpenSocial approach, where applications exist on a site rather than off of it

Solution 4: Create off-site APIs and activities
To some extent, this can happen by itself with an API or not, as passionate users will create forums, mailing lists, blogs, and other social structures about your product. But as Twitter and blogs show, you can build an API which allows off-site applications and websites to build richer functionality. It will be interesting to see if Twitter eventually creates an on-site API a la Facebook, or if they will always make their onsite experience very simple and clean.

Solution 5: just focus on one thing
And the final solution is just to ignore your users, and focus on the main value that your product provides. This certainly has a nice charm to it, but obviously few companies follow this – more ambition leads to more features, typically, even though the user experience might suck as a result.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

Planning for success, not failure

High achieving people who have a long history of being successful often plan accordingly – doing so, of course, means that they plan for success in whatever they do. And when you take a successful person and put them in a successful big company that’s already making money from their products, there’s even more reason to plan for high-achievement outcomes.

But let’s say that you put these successful people and put them in environments of great uncertainty, like at a Silicon Valley startup – what happens? That’s when realities collide! When you apply the big successful company playbook to startups, you can end up with monolithic planning processes, products that can’t find their markets, and lots of money being spent on launches for the wrong products. It’s not that these tactics are stupid, it’s just that they don’t work as well when you’re dealing with ill-defined customer problems with unknown solutions.

At the heart of this conversation is – what happens when you take something that’s usually assumed to be successful, and you instead say that it’s very likely to fail?

In a way, you can think of this as planning to fail, but then building the support structure around the failure in order to create a failure-tolerant system. Let’s dive into this.

Planning for failure, not success
The title of this blog refers to the fact that companies like Google, IDEO, and 37signals all have the culture of “Failure is OK” built into them.

At Google:

• Google makes money by being always available, ubiquitous, and having a great product
• To deliver their service, they have 100,000s of servers (maybe more?)
• Any one of these servers have a high likelihood of failing at any time
• To create a fault-tolerant system, they have lots of redundancy and lots of sophistication around what happens when an individual box fails
• Contrast this to a big-iron approach that builds all the redundancy into specialized hardware that’s designed to never fail

At IDEO:

• Companies hire IDEO to give them fresh designs based on a customer-focused approach
• Part of every project involves lots of brainstorming and coming up with ideas
• However, any specific idea is likely bad (for example, 12 out of 4,000 toy ideas were actually successful = 0.3%)
• Thus, IDEO combines structured brainstorming, rapid prototyping, and field research to rapidly try out new concepts and get to good products
• Contrast this to a process where the “Great Man” designer thinks about a design problem and then comes up with the right solution spontaneously

At 37signals, in particular Ruby on Rails:

• Rails is framework built for programmers to build websites
• Of course, every web project requires lots of lines of code which can easily break at any moment
• If you assume that programmers will more often write code that is buggy and breaks, then you’ll want to make testing and iteration easy – this is at the heart of Agile, TDD, continuous integration, and other related disciplines
• Contrast this to a waterfall engineering approach which assumes the correct design and architecture can be thought out by experienced software engineers

Each one of these examples is similar, yet unique in their own way – but there are similar themes that pervade each one of these approaches.

Characteristics of failure-tolerant systems
Each one of these systems takes the central part of a process and assumes failure, and then builds up a support system around it.

This happens by building on a few core principles:

• Acceptance of failure: You have to accept that shit happens and failure is commonplace – this needs to be internalized so that failure isn’t punished, but rather embraced!
• Massive redundancy: Then, it needs to be easy to have lots of redundancy built into the system – for designers, that means lots of designs get generated. For startups, that means lots of ideas are tested, and for Google, that means lots of servers are used
• Cheap, easy, fast: As a side-effect of the redundancy, it needs to be easy, cheap, and fast to have lots of ideas, lots of servers, or write lots of code. The harder it is, harder it will be to create redundancy
• Iterative, reality-based testing: Testing these individual components constantly becomes key – you need to force failure on the system to figure out how it reacts from a system-wide level

Building up processes based on the ideas above makes it easier and easier to deal with failure and come out on the other side!

Conclusion and next ideas
There are lots of interesting directions that this line of thinking can go.

This area of thinking started out with the hiring process, and the idea that maybe interviews don’t work at all – there’s a bunch of academic research that implies that, actually. So if how would you build a failure-tolerant system around the hiring process, if you assume that good interview candidates actually have no correlation to successful employees?

For dating, what happens if you assume that people you like to date may not be the kind of person you’d have a successful marriage with? What if people suck at figuring out what kind of guy or gal is the “type you’d bring home to Mom?” I think anyone could attest to the idea that many people suck at figuring out the right person to date, much less the right kind of person to marry. I personally find it crazy that people make a 50+year decision to be married based on a 18-month sample size :-)

For careers, what if it turns out that people have a really bad idea figuring out what they’ll actually want to do 40 hours a week, 50 weeks a year, for the rest of their life? How would you figure out the right career faster rather than shorter?

All of these are great thought experiments, I think.

What else am I missing? :-) I’d love to take any suggestions and write up some thought experiments around it.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

I’ve been auto-posting all my tweets on my blog. I’m curious what people think about it – useful? Or not?

Take the poll below, and please leave comments if you have extended thoughts:

Thanks for the feedback!

The perils of ignoring brand value
The nature of internet marketing makes it easy to have a highly accountable, metrics-driven view – but companies that are highly metrics driven easily overlook hard-to-measure issues like brand and user experience. The reason is that when all product decision-making is run through metrics-driven reports, soft things like “Brand” show up as costs, but never as benefits.

This leads to systematic erosion in many “soft” but important factors, like customer experience, brand value, and “love.” :-) And ultimately you need all of these things to create a massive, enduring consumer brand – it’s not enough to optimize funnels.

Let’s discuss why:

Two worlds: Direct marketing and brand marketing
In the advertising industry, there’s been a long, historic distinction between brands and direct response – and this distinction echoes its way into the online startup building world as well.

In the brand world, you have companies like Coca Cola, Apple, and others who pour millions of dollars into high-reach vehicles like TV which lack any real accountability. Thus the saying:

Half the money I spend on advertising is wasted; the trouble is I don’t know which half
— John Wanamaker, US department store merchant (1838 – 1922)

To many people, the brand advertising world is irrational and fashion-driven, because of the complex interactions between agencies, their partners, and the publishers that rely on them. Just watch Mad Men.

On the other hand, you have direct marketers who thrive on accountability. They buy into marketing channels like direct mail, coupons, infomercials, and most recently online remnant ads, because they can purchase cheaply and use sophisticated statistical techniques to optimize their media buys.

Startup engineers tend towards metrics-driven
So what side do startups tend to side on? It obviously depends, but because of the highly accountable and measurable nature of online, it’s much easier to become metrics focused. Similarly, startups are mostly poor ;-) Thus, expensive brand efforts are mostly out of reach. (Probably for the better!)

Also, with the possible exception of GoDaddy, I don’t know a single startup that made it or not based on their brand advertising strategy. The typical path is focused on products and technology, and large organic growth which builds large consumer audiences.

And obviously, readers of this blog will tend to be much more metrics driven compared to the average entrepreneur!

You optimize what you measure
The first issue that causes metrics-driven startups to ignore brand value has to do with the fact that it’s very hard to measure brand, and you tend to optimize what you can measure. As soon as you throw some numbers on a big report, there’s an inherent human desire to make the numbers go up!

This is why one of the fundamental tenants of metrics-driven startups is to build lots of highly accessible reports that everyone in the organization can look at. Even if it’s easy enough to pull something out via a SQL query, it’s another thing for everyone to be able to hit a URL and load it instantly, no matter who they are on the team.

Measuring brand value is hard!
But measuring brand value, or user experience, or community “feel” or other soft things like that is very hard. I think they’re hard because while it’s clearly important, at the same time:

• The quantitative effects accumulate over large periods of time
• These might be “source” variables that drive lots of behavior, but it’s hard to measure past surveys and explicit information collection
• Some of the most important datapoints may be qualitative, not quantitative
• Changing these soft things may require big efforts above and beyond small A/B-testable changes

The companies out in the marketplace that try to measure brand value mostly just use surveys to detect changes. Or, many companies simply resort to a pretty ineffectual number like “reach,” which refers to the number of people who saw the campaign. This can sort of work, but self-reporting also sucks, and the quantitative data you get out may not be as useful as the qualitative data.

In my previous online ad career, I was shocked to hear that the standard way to measure a brand advertising campaign online was to fork $50k over to Dynamic Logic, whose job was to run a dinky little survey and tell you if your campaign worked. $50k to run a survey!

Reports show the cost of branding, but not the benefits
As a result of brand advertising being hard to measure, you get two systematic, interrelated issues:

1. Product changes that result in brand value are overlooked, whereas the costs of delivering that value is not
2. Features that negatively impact brand value but show short-term quantitative value are accepted

Here are two examples – let’s say that you think your site’s interface looks like crap, and you want to improve it to make it higher class and more trustworthy. But your metrics czar says, let’s make a really small improvement and see if it affects anything before we revamp the whole site. That sounds reasonable, but then you find out that in fact, making a visually compelling site just doesn’t drive better metrics, and in fact, it’s expensive and maybe lowers certain metrics. What do you do? (This is case #1)

Another example is that you make it really hard to unsubscribe from your mailing list. Maybe you don’t have a link, or you have to login first, or whatever. Making this change clearly affects your ability to retain users, but you get a small percentage of complaints, but the overall quantitative metrics look good. Should you keep this hard-to-unsubscribe mailing list issue? (This is case #2)

Ultimately, it should be clear that both cases are not clear cut issues at all. I could find reasons to go either way, but when you’re trading off a qualitative metric versus a quantitative thing, the numbers-driven approach tends to win. But this may not be the right thing. Similarly, sometimes the numbers may justify the decision, and the brand costs are actually quite low.

How do you make these decisions then? I’ll just wave my hands and say, “Entrepreneurial judgement” ;-)

Who’s the brand advocate?
One of the big, important roles that you need on every team as a result is someone who can advocate for the soft things. Who’s your brand advocate? Or customer experience advocate? Having someone on your team who can make logical arguments to balance out the quantitative stuff is hugely key, otherwise you’ll inevitably go down a path of brand-eroding quantitatively driven decisions.

Similarly, if you find that you’re never making decisions that go against the numbers, then frankly, you’re probably doing something wrong. If the data drives all the decision-making, then a lot of “soft” data is getting ignored.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

Don’t worry, I’m not a hippie
From the title of this blog post, you might think that I’m going to make a touchy-feely argument about why you should respect the right of your users to do all the terrible things that every entrepreneur fears:

• delete their accounts
• unsubscribe from email lists
• cancel their subscriptions
• uninstall their apps

… but you’d be wrong.

In fact, I’m going to argue that for every early product out in the market, making it really easy to quit is completely aligned with self-interested thinking. I’ll make the assumption that all the entrepreneurs reading this post are greedy, self-interested individuals, and target the appeal straight into your dark hearts ;-)

My central argument is that if you believe that every startup is an iterative learning process that converges towards product/market fit, then you need extremely high-fidelity signals to tell you if you’re going in the right direction. That means that along with trying to charge people money from early on, which is the highest form of “I love this!” you should give people valves to tell you “I hate this!” so that you can learn more faster.

Let’s drive into this further…

Product/market fit
There’s a notion of product/market fit that Marc Andreessen references in his blog, and he calls it the “only thing that matters” and says that every startup should do everything they can to get to this point. Let’s see what he writes:

The only thing that matters is getting to product/market fit.
Product/market fit means being in a good market with a product that can satisfy that market.

… and Marc continues:

Lots of startups fail before product/market fit ever happens.

My contention, in fact, is that they fail because they never get to product/market fit.

Carried a step further, I believe that the life of any startup can be divided into two parts: before product/market fit (call this “BPMF”) and after product/market fit(“APMF”).

When you are BPMF, focus obsessively on getting to product/market fit.

Do whatever is required to get to product/market fit. Including changing out people, rewriting your product, moving into a different market, telling customers no when you don’t want to, telling customers yes when you don’t want to, raising that fourth round of highly dilutive venture capital — whatever is required.

When you get right down to it, you can ignore almost everything else.

If you believe what he says, that gives you a pretty firm set of marching orders. And for early products on the market, getting to to this point in which your product is good enough and the market is compelling enough is a tough slog. So the question is, how do you navigate your way to product/market fit?

At the heart of every startup is a learning loop
For the idea that every startup is inherently a learning machine, we can turn to two of my favorite startup people, Steve Blank and Eric Ries. Eric has blogged in a lot of detail about how he believes that inside of every startup is an OODA loop that involves trying stuff out, learning, and trying more stuff again. And of course a lot of these ideas are built off of Steve Blank’s Customer Development framework that I’d encourage my readers to look into as well.

In this light, to combine the two ideas: Every startup is a series of iterative experiments that gets you from zero to product/market fit, and if you can do it before running out of money, then you might get rich ;-)

And the decision-making process in this approach is totally different. In most product strategy conversations I’ve been involved in, the most heated debates center around whether a particular product will work, and all the pros and cons of the situation. Contrast this to a learning-centric approach, which emphasizes whether or not experimenting with an idea will yield insights, and how much it’ll cost to learn these insights.

In other words, you’re much more likely to try things that will fail, if those failures teach you something important about the market.

Of course, all of the decisions that power these iterations rely data – and the better the data, the better your decisions will be, naturally. So where do you get the data to tell you if customers are happy or not about your product?

Explicit signals beat implicit signals almost every time
One of the key lessons I took away from my time from the behavioral targeting ad industry is that explicit data is much, much better than implicit data, when it comes to predicting user behavior.

That is, you’d prefer explicit “intent” data like:

• made a purchase
• used a student loan calculator
• searched for “palo alto bmw dealership”
• filled out a form

versus the less valuable implicit “interest” data like:

• have similar demographics to other people who buy
• visit the same publications as similar customers
• having a pattern of reading finance articles

So if you are looking to collect data to drive decisions, then the best kind comes from the explicit data of having users specifically take action, whether it’s positive or negative. Purchase intent data, as illustrated above, is positive – and quitting intent gives you the negative half. In fact, if you only look at the positive feedback, you might be ignoring 50% of your data.

As a result, you want lots of explicit data points in the axis of “I love it!” to “I hate it!” which includes people giving you money (maybe donations being the ultimate form of love) to allowing them to easily quit. Make it easy  for your users to quit, unsubscribe, or otherwise cancel – it gives you the strong signal when you’re doing wrong! And make sure to track it and include it in all of your quantitative experiments as well.

Better data = better learnings = Better product
So to summarize my key arguments here:

• Give users lots of explicit ways to show appreciation and hatred
• These datapoints will help you iterate your product
• Better product iterations will let you reach product/market fit faster
• Reaching product/market fit will lead to more money faster

You can only learn so much from reacting to positive data, and trapping your users in unwanted subscriptions won’t get you to product/market fit any faster.

And finally, don’t do it because it’s annoying ;-)
‘nuf said.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

Measuring value created rather than optimizing for yourself
In my last blog post, I talked about the idea that value creation generates revenue, traffic, and other metrics, not the other way around. This is a particularly interesting idea to implement because it goes against much of the standard analytics reports that are out there.

The reason is that ultimately, most metrics tend to focus inwards, on self-interested gain, rather than outwards on the value you’re creating for your customers. Let’s take a couple examples of inward-focused metrics that people often cite:

• account registrations
• pageviews
• unique visitors per month
• revenues

I’m sure you measure many of the above, as I do as well. It’s OK to measure this stuff, but if you start to optimize for it, you are starting to focus on the business of value extraction, not value creation.

Measuring # of life preservers sold versus the # of lives saved
Thus we come back to the title of the post. Most people are using standard analytics packages that are commoditized to focus inwards on metrics like pageviews and revenue. To use an analogy, that’s akin to the idea of measuring the # of life preservers that you sell, and trying to optimize for that, rather than optimizing for the benefit, which is the # of lives saved.

If you focus primarily on selling life preservers, then you’ll tend to do all sorts of stuff like:

• making them cheaper to build
• adding doo-dads to them that are flashy to customers
• using aggressive sales tactics
• etc.

These things might generate revenue in the short to medium run, but if you prioritize this at the expense of actually delivering on the product benefit, then that’s a bad optimization in the long term.

Now contrast that to the idea of trying to save as many lives as possible. You might still want to make them as cheap as possible, so that every ship in the world can have as many of them as needed. You might still want to add upgrades to them, but only if they help save lives. etc. While these changes may be similar in execution, they are different in spirit than the changes you’d make when optimizing for sales.

Introducing “Benefit-Driven Metrics”
So ultimately to start this exercise, you should throw out all the standard metrics (conversion rates, pageviews, etc.) and just focus on one thing:

What are your customers measuring?

By looking at how they define value, then you get yourself aligned to them as closely as possible. Answering this question sets your company up for value creation, which then unlocks the ability to gain something from that value, then you have to start here.

I’ll deem these quantitative measurements as “Benefit-driven metrics.”

How do you measure it?
Here’s the interesting part – everyone’s benefit-driven metrics will be completely different, because most people’s customers and value proposition and product are ultimately very different. Unfortunately, you don’t have the crutch of standardized numbers like pageviews or uniques to lean on.

But let me give you some examples for reference:

For dating sites:
Why do customers join dating sites? To find their soulmates. Thus, measure the quantity of successful matches you make, not the lifetime value of the customer. Focusing on LTV can easily lead you to do things like creating fake accounts to make people come back, or optimizing it so that they find their best matches several months down the line, or trying to get everyone to pre-pay for the service rather than making the product experience awesome.

For marketplaces:
Why do customers sell on a marketplace? To make money and get rid of their stuff. Why do customers buy on a marketplace? So that they can get things cheaply and quickly, and are happy with their purchase. Thus, measure the quantity of how much your sellers take home, and how many buyers are happy with their experiences. Contrast this with overfocusing on listing fee revenues, which might get you into a spiral of raising prices rather than creating the best commerce experience.

For social networks:
Why do customers use social networks? To “connect” with their friends – let’s boil that down to communicating (though it’s obviously much richer than that). Then ideally, you might want to focus on the number of messages/comments/posts that end up getting replies from their friends. If you overfocus on something like user registrations, then you might get a ton of users, but maybe they won’t be getting to experience the benefits of the product.

For online publishers who sell to advertisers:
Why do advertisers buy ads on websites? To generate traffic to their own sites, which in turn leads to revenue. In this case, you should quantify the amount of revenue you generate for your advertiser customers, or at least the number of conversions they receive. Contrast this with the approach of measuring and optimizing your own CPMs as a publisher, which results in potentially delivering a lot of crappy traffic to advertisers who will drop their payments in the long term.

Special note for ad-driven startups :-)
Now ad-supported startups have a particularly interesting issue in this, because I keep using the words “benefits” and “customers” and perhaps it’d be easy to think this refers to the users of the product. But maybe not, as I’ve outlined before in Your ad-supported Web 2.0 site is actually a B2B enterprise in disguise. The reason is that your customer may actually be the advertiser on the site, not your user!

And in fact, if you overfocus on pleasing your users to the detriment of your advertiser customers, which is very easy – then that leads to very bad things.

Start this benefits-driven approach now, not later, so you can learn the right things
Finally, I want to emphasize that I believe it’s important to start thinking about these benefit-driven metrics from the beginning of your business, not later. The reason is that every learning that a startup makes is often hugely applicable to a specific context, but not at all applicable to other variations.

If you’re going to start a website that churns users like crazy but hits massive user goals, you will build an entire organization to optimize for those metrics. And once you’ve gone far on this, it’s not clear that you’ll have the DNA, the technology, the ideas, or the willpower to execute in a different direction.