Andrew Chen Archives

Here are some links I’ve posted to my twitter account over the last week or two. You can follow me on Twitter if you like these! Many are work unrelated.

• lol, lots of ex-softies (located in kirkland goog office) complaining: http://tinyurl.com/a35rm5 
• this article on pitching game publishers reminds me of articles on pitching VCs: http://www.icopartners.com/blog/archives/33
• i’m a big ARG skeptic, good to see this article: everything you know about ARGs is wrong – http://bit.ly/BM2A
• zynga’s new MMOG – highest production facebook game so far? http://bit.ly/WZMP 
• interview with an engineer who wrote adware – lots of interesting tidbits in there: http://bit.ly/g8v9 
• neat (but oldish) – detailed analysis of second life economics: http://tinyurl.com/5z5j3u 
• great blog post on why NYT landing pages suck. Guys, this is pretty basic, get it together!… http://bit.ly/10sVq 
• generation Y research – this stuff is usually too fluffy for me, but sometimes a good nugget here or there: http://bit.ly/89Ub 
• great article on the lookery blog about behavioral targeting:http://tinyurl.com/82vmtk 
• very readable fraud confession from CEO of satyam:http://tinyurl.com/8myeob
• wow, actually found a good blog on freemium: http://www.freemium.eu/ 
• wow did you know google chrome automatically updates itself, whether you’re running it or not? http://bit.ly/1hkqeQ 

The 2009 conference season begins
Happy new year! Recently, Jason Oberfest of MySpace put together a great resource of a bunch of digital media tech conferences. I added a couple ad-related conferences, and he agreed to graciously share.

Is this list complete?
Of course this list isn’t complete at all. If there’s anything missing in the list, please leave a link in the comments!

The digital media conference list (so far):

Demos and General Technology

• SXSW
• Startonomics
• Under the Radar
• The Crunchies
• Foocamp (invite only)
• AlwaysOn
• TechCrunch 50
• Web 2.0 Expo
• DEMO
• Le Web (Europe)
• The Next Web Conference (Europe)
• Lift (Europe)

Platforms, apps, and widgets

• Graphing Social Patterns
• Google Campfire
• Widget Summit
• SNAP Summit

Online Advertising events

• Advertising Week
• OMMA
• MIXX
• DMA
• Ad:Tech
• Search Engine Strategies
• Search Marketing Expo
• Pubcon
• Affiliate Summit
• eMetrics
• Leadscon

Gaming

• Game Developers Conference
• Virtual Goods Summit
• Social Gaming Summit
• iGames Summit (iPhone games)
• Interplay
• Casual Connect
• Flash Gaming Summit
• LOGIN
• Engage! Expo

Business and analysts

• Web 2.0 Summit
• All Things D
• EconSM
• MIT Emerging Technologies Conference
• Goldman Sachs Internet Conference
• Fortune Brainstorm Conf

UPDATED: added a couple games and search ones, courtesy reminders from Jameson Hsu, Joe Ludwig, Lee Clancy, Wallen’s, Niki Scevak Charles Hudson, Andrew Parker, and Jonathan Mendez. Thanks!

Still on blog vacation – hopefully not for long
Sorry for the lack of posts guys – still semi on blog vacation ;-)

Anyway, I will be in Seattle on the 8th and 9th of January – if anyone wants to meet up while I’m there, here’s where I’m going to be:

• 8th: Downtown Seattle (free after 5:30pm)
• 9th: Downtown Bellevue (free after 5:30pm)

If I get any interest, I’ll set up drinks or meet people for coffee or something. Let me know, just shoot me an email at voodoo [at] gmail.

See you there!

A case study for the Freemium business model
There’s been a lot written about the Freemium model over the years, particularly from Fred Wilson at AVC. Here’a couple articles I’ll recommend on the topic:

• Fred Wilson: My Favorite Business Model
• Sean O’Malley: Will Freemium save Web 2.0?

Anyway, given the great interest in direct monetization due to the economy, I decided to write a post on the topic as well, focusing on AdultFriendFinder, which has recently released a bunch of great data.

AdultFriendFinder files for IPO
Recently, the holding company FriendFinder Networks, Inc., the owner of AdultFriendFinder (and Penthouse), filed to go public. As such, they released a Form S-1 where they go through many of their overall business metrics – you can read the full SEC documents here.

OK, it’s a lot more boring than you might expect from an adult-oriented conglomerate. It’ll put you to sleep.

There’s a ton of data in the S-1, but I’ll just summarize some of the most interesting stuff related to acquisition, monetization, and retention of the customerbase…

FriendFinder’s business metrics
First off, there’s some good definitions for how they think about their business – everyone who is thinking about direct monetization of their customers should be familiar with all the terms below:

• Visitors. Visitors are users who visit our websites but do not necessarily register. Visitors come to our websites through a number of channels, including by being directed from affiliate websites, keyword searches through standard search engines and by word of mouth.
• Members. Members are users who complete a free registration form on one of our websites by giving basic identification information and submitting their e-mail address. Members are able to complete their personal profile and access our searchable database of members but do not have the same full access rights as subscribers.
• Subscribers. Subscribers are members who purchase daily, three-day, weekly, monthly, quarterly, annual or lifetime subscriptions for one or more of our websites. Subscribers have full access to our websites and may access special features including premium content.
• Paid Users. Paid users are members who purchase products or services on a paid-by-usage basis.
• Average Monthly Net Revenue per Subscriber. Average revenue per subscriber, or ARPU, is calculated by dividing net revenue for the period by the average number of subscribers in the period.
• Churn. Churn is calculated by dividing terminations of subscriptions during the period by the total number of subscribers at the beginning of that period.

And in fact, you can think of Visitors, Members, and Subscribers/Paid Users as a funnel that they manage to extract revenue. Then you combine that with the ARPU and Churn to get an understanding of monetization and retention, respectively.

Funnel metrics
And in fact, they list some of the key conversion rates between each of these numbers:

• Visitors: 59 million uniques worldwide
• Members: 4 million new member registrations (on 270 million members total)
• Subscribers*: 900k paying subscribers
• ARPU: $19.06 per paying subscriber/paid user
• Revenue per member**: $0.95 per member
• Churn: 18% month over month

*New subs/month number is not in the S-1, but is derived from the fact they have been about level on total # of subscribers, yet with 20% churn, so to make that up they must be adding a little over 200k to stay even.

**Revenue divided by new members (rather than subscribers)

So let’s try to figure out what their funnel %s look like. One complication is that the FriendFinder S-1 gives total members and new members within a month, but doesn’t give “active members.” At the minimum, it should be 6% since 4 million new members on 59M unique visitors yields 6%, but you might imagine it would be closer to 15%, which is closer to an industry standard.

Here’s a couple ranges, based on the above assumption:

• Visitors -> Members: 6-15%
• Members -> Subs: 10-22%
• Subs -> Renewing Sub: ~80%
• Revenue per member: $0.48-$0.95

You can compare this to Free-to-play games and Casual MMOGs via these blog posts from Jeremy Liew and Nabeel Hyatt, although you should note that their definition of ARPU is calculated from actives rather than subscribers, so you should use my above “Revenue per member” number of $0.95 rather than the substantially higher $19.06 per paying subscriber. Anyway, you’ll see that it’s actually not much different!

It shouldn’t be surprising that all of these freemium models, after careful optimization, come out at roughly the same numbers.

Acquisition
I also want to highlight a couple interesting points about how FriendFinder acquires their customers:

Marketing Affiliates. Our marketing affiliates are companies that operate websites that market our services on their websites. These affiliates direct visitor traffic to our websites by using our technology to place banners or links on their websites to one or more of our websites. As of September 30, 2008, we had over 110,000 participants in our marketing affiliate program from which we derive a substantial portion of our new members and approximately 44% of our revenue. For the nine months ended September 30, 2008, we made payments to marketing affiliates of over $46.4 million.

[also, in a separate section…]

In addition, for over 10,000 of our affiliates, we maintain private label websites that provide a seamless, turnkey outsourced solution using our technology platform for social networking and live interactive video websites. These websites have the look and feel of the affiliate’s website with the affiliate’s logo and website name but are operated by us. Users who click through the affiliate’s website are tagged with the affiliate’s identifier that tracks the user to calculate the payment due to the affiliate. Private labeling allows our affiliates to preserve their brand while generating revenue for us.

There’s also a block of text on the extent to which they spend money on Search Engine Marketing:

We rely on both algorithmic and purchased search results, as well as advertising on other internet websites, to direct a substantial share of visitors to our websites and to direct traffic to the advertiser customers we serve. If these internet search websites modify or terminate their relationship with us or we are outbid by our competitors for purchased listings, meaning that our competitors pay a higher price to be listed above us in a list of search results, traffic to our websites could decline.

How much do they spend on SEM? Turns out it’s over $50MM per year:

The largest single selling and marketing expense item for Various were “ad buy” expenses which amounted to $54.8 million and $55.3 million for the 2007 period and the year ended December 31, 2006, respectively, the cost of purchasing key word searches from major search engines, together with expenses related to associated personnel.

This means that between their affiliate payments and SEM, they are spending around $100MM buying traffic!

Conclusion
There’s other neat info in the S-1 about their chargeback rate, the split in revenues between their magazine side (Penthouse) versus AdultFriendFinder, and also the breakdown of all the smaller demographics they service. For example, did you know that they also run a bunch BigChurch.com for Christian singles?

Anyway, there’s lots of fun information in there – if you find something fun in there, let me know and I’ll update this blog post to reflect it.

As always, comments and questions are welcome!

I’m still on blog vacation but I figured I would reblog the links I’ve posted to my twitter account over the last week or two. You can follow me on Twitter if you like these! Many are work unrelated.

Anyway, here we go:

• Richard Dawkins and Derren Brown talk about “psychics” – fun set of videos: http://bit.ly/XTbB
• on-the-ground report of the startup scene given the recessionary environment from an ex-googler: http://bit.ly/BWKv
• presentation zen: nice lil video, shot all on cell phone, wins awards: http://bit.ly/6sVJ
• Will Freemium save Web 2.0? http://tinyurl.com/6p5ejq
• programming language that looks like cooking recipies: http://www.dangermouse.net/esoteric/chef.html
• mobile CPMs collapsing, says AdAge: http://tinyurl.com/6p8d8j
• obama logos that weren’t chosen, some are still quite good! http://tinyurl.com/5c4bsj
• emarketer predicts down revenue year for myspace and facebook in 2009, 585M and $210M respectively: http://bit.ly/UVDj
• michael lewis (author liar’s poker) on jobs versus callings and a decimated wall st: http://bit.ly/zV8s
• hilarious basketball trick shot competition where lebron james loses: http://tinyurl.com/3qrtcg
• this kind of blog post makes me happy i’m not at msft: http://tinyurl.com/5so6ok
• New Malcolm Gladwell article: How do we hire when we can’t tell who’s right for the job? http://bit.ly/wm29
• great article on why there isn’t more transparency in the leadgen market: http://bit.ly/YB41
• watch a genetic algorithm try to evolve a car in flash, traverse the land without red circles touching the ground: http://bit.ly/fLgb
• creative! 3-d dominos sandbox: http://drawminos.com/
• comic: mostly i use facebook to send cutesy, useless gifts that require people to install apps that email them spam http://bit.ly/xEff
• funny video – chickens stop rabbit from fighting: http://tinyurl.com/2m5227

BRB!

Here are some of the partially-written drafts I have saved in my WordPress account – no promises on whether or not I’ll finish them though :)

• Forget about user engagement – just focus on retention!
• Launching new product features: Metrics for internal distribution
• The Joel Test for data-driven decisions
• Do data-driven models lead to uninspired products?
• Are websites really better than desktop apps for user acquisition?
• The social gaming business 101: Lifetime value, cost per acquisition, player retention and lifetime value
• 5 key factors for increasing retention metrics
• Dual currencies in virtual economies
• Elements of every virtual items-based social game
• How to calculate an ARPU and why it’s important
• 5 common objections to making data-driven decisions
• “Yet another ad network?” Why the future of advertising points at more networks, not less
• Are you building features that no one will use?
• “OMG I’VE JUMPED THE SHARK”: 10 things to do after the TechCrunch crowd has moved on
• Learning about retention metrics from retail and catalog marketers
• Designing social networks: Real friends versus online friends
• Social gaming and BBS door games: Learning from past casual asynchronous games
• Why “launching” is old and busted

See you guys in a couple weeks…

Virtual Goods Summit videos are up!
Charles Hudson was kind enough to post all the videos from the Virtual Goods Summit from earlier this year, you can see all the videos here.

The talk that Daniel James (of Three Rings fame) and myself gave is embedded below.

Here’s the outline of the content we covered:

Metrics for Virtual Goods

• Key metrics
  • What is Whirled?
  • What is Puzzle Pirates?
  • Puzzle Pirates Metrics
• What factors drive LTV?
  • User acquisition metrics
  • Factors that drive acquisition cost
  • Dashboard for user acquisition
• Customer retention metrics
  • Factors that drive revisit rate
  • Whirled retention cohort %s
• Virtual Economy overview
  • Dashboard for a virtual economy
  • Lots of graphs
• Billing payment breakdown
  • Billing fraud
• Questions and answers

Enjoy!

UPDATE: Mike Gowen put up some photos of the slides here. Mindmaps and more videos from Jussi here.

Buying ads make sense for direct monetizing products
When it comes to products that directly monetize their audience using subscription, ecommerce, virtual items, etc., it can make a lot of sense to rely on advertising as a distribution channel. The reason for although there are 100s of millions of internet users, only a small fraction (usually <1%) will be in-market for your services at any given time. As a result, you are looking to acquire these users, and only these users, and everyone else is considered wasted energy.

Note that the freemium model is a variation on this concept – where you acquire a large base of “casual users” that stick around, and you slowly convert some % of them to subscribers. You can think of it as vertically integrating your distribution, and instead of spending money to buy ads, instead you are spending money to support this large base of free users.

Run an ad-supported consumer internet app? You better understand ad-buying too
These days, it’s simple to think that advertising is about placing javascript code on a page, and seeing what numbers AdSense gives back to you. Instead of thinking about it that way, publishers would do their customers (their advertisers) a great service by understanding what it means for ad inventory to perform or not perform. They should really understand how advertisers analyze their cost-per-acquisition so that the publisher can better service them – that’s at the heart of indirect monetization. If you’re not selling directly yourself, you’re helping someone else sell.

CPA, the common currency of user acquisition
The first step is to understand your user acquisition funnel, from start to end. Although there are many ways to price things, be it CPM, CPC, or CPA, the key is that it all rolls back to how much it costs you to have a registered user. You need this cost-per-acquisition number to be lower than the lifetime value number, and what you have left is profit (before cost of infrastructure, etc).

So you want to build something that looks like this:

The above is an example of two traffic sources, Google and Advertising.com (the latter being an ad network), as well as clickthrough rates, signup %s, and the cost per acquisition.

A couple important notes on the above:

• the SOURCE of your traffic is the most important segmentation – make sure you track acquisition and LTV numbers, since you often get vastly different numbers depending on where you are buying ads
• you want to break down your funnel into as small of steps that make sense, from the clicks into the signup page into any intermediate profile forms and then the final registered numbers. Your funnel may be larger or smaller
• Google might charge you CPC and Ad.com might charge you CPM, but you have to normalize that back into how much it costs you to acquire a registered account. In a CPC model, you don’t care about the CTR much since you don’t pay for impressions that don’t result in clicks, whereas you do care about CPMs
• the only difference between a good CPA and a bad CPA is whether it’s above or below your customer LTV
• In addition to tracking source of traffic, you may also want to track important factors like what campaign it was in, what creative it corresponded to, the banner ad size, and other things that might affect CPA. The last thing you want is a variation that is very unprofitable, but is obscured by being grouped together
• You may also want to group all your marketing channels into the above, including email, partnerships, blog traffic, viral invites, etc. Obviously for stuff that’s free traffic, the CPA is infinity, but it’s good to know what kinds of funnel %s the other traffic throws off, for comparison’s sake

Got all that? Good :)

What factors influence ad performance?
The second thing worth considering is what factors actually influence GOOD numbers for CPAs versus what numbers are generally bad.

I made a quick, anecdotal table below to enumerate some of the factors:

A couple additional notes:

• As mentioned previously, the source of traffic is very important – you should dedicate a significant amount of time buying lots of different kinds of traffic to see what works
• Cost model is something you should be able to normalize into CPA and mostly ignore, except for cashflow and risk reasons
• User requirements can be a huge issue – if you are forcing users to download, that will kill your CPA. Similarly, asking a demographic that doesn’t have credit cards for their credit number can kill you. Make sure that you understand your audience and that your funnel is optimized for them
• Audience Theme revolves around the concept that strongly themed products are often quite vertical in nature, which causes a large % of users to reject the product. For example, a site for teens obsessed with vampires is much narrower than a web email site. The narrower the theme is, the harder it is to find appropriate ad inventory to buy
• Funnel design and A/B testing is key – definitely worth investing in
• Similarly, for those who can find a viral angle in their product, that can be a huge benefit as well. It has the capability to create order-of-magnitude decreases in the CPA, which can be the difference between profitability and bankruptcy! But this also has issues if your product is not widely appealing enough, since virality depends on a horizontal offering to work

If you have questions or comments, feel free to leave a message!

Man it’s tough to make money with affiliate sales :) There’s definitely been a bump since I posted my books list, but nothing fancy.

This is why I’m always amused when I get pitched blog widgets that are supposed to help me make money – none of it is ever substantial enough to be worthwhile purely for monetization. The value from writing this blog is all in super soft social capital.

Talk coming up next week
Steve Blank, a professor at Stanford and serial entrepreneur of 8 startup companies, is giving a revised version of his talk, The Secret History of Silicon Valley. The YouTube video of the talk is embedded above, and an additional 1/3 of new material will be added. There’s also a lot more material in Stanford’s Entrepreneurship Corner, which I’d encourage you to check out aswell.

Also, if you haven’t read Steve’s book, The Four Steps to the Epiphany, I’d encourage you to do so. There’s a lot of great stuff in there that is more or less the product development methodology that I follow.

Anyway, here are some additional details on the event – see you there!

DATE & TIME
Thursday, November 20, 2008

12 p.m. – Bring your lunch and enjoy a lecture with CHM friends and family. Beverages will be provided.

LOCATION
1401 N. Shoreline Boulevard
Mountain View, CA 94043
Directions

REGISTRATION
Register Now

ABSTRACT OF TALK
While Silicon Valley is responsible for the wealth of millions of people, not many are familiar with its long and complex history. Unbeknownst to even the most seasoned inhabitant or observer, Silicon Valley, Northern California’s peninsula, was shaped by many forces.

Join renowned serial entrepreneur, Steve Blank, as he provides an overview on the secret history of Silicon Valley and how the Valley got its start. Much like the startups that have made Silicon Valley famous, the Valley began in a strikingly similar formula.

Hear the story of how two major events – WWII and the Cold War – and one Stanford professor set the stage for the creation and explosive growth of entrepreneurship in Silicon Valley. In true startup form, the world was forever changed when the CIA and the National Security Agency acted as venture capitalists for this first wave of entrepreneurship. Learn about the key players and the series of events that contributed to this dramatic and important piece of the emergence of this world renowned technology mecca.

I recently went through and updated the list of essays on this blog – it’s now all up to date. As always, you can view them from my blog by clicking the Essays link at the top.

For the lazy folks, here’s the complete list, current as of today:

Viral marketing and user acquisition

For web entrepreneurs, growing your userbase is a key challenge, alongside product development and financing. These posts emphasize a quantitative approach to getting traction and growing users.

• What’s your viral loop? Understanding the engine of adoption
• Adwords is not enough for success on the consumer web
• Are your SEO efforts working, or failing?
• Bridging your traffic engine with your revenue engine
• Facebook viral marketing: When and why do apps “jump the shark?”
• Is your site really viral? Viral Branding versus Viral Action
• Social network marketing: Getting from zero to critical mass
• Viral marketing is not a marketing strategy
• 10 obvious strategies to ruthlessly acquire users
• Growing renewable audiences (a talk at O’Reilly Alphatech Ventures)

Engagement and product design

Using principles from game design and analysis of consumer behavior, these essays cover the process of creating experiences your customers will love.

• 10 signs you’re a product fanatic
• 25 reasons users STOP using your product: An analysis of customer lifecycle
• Are people like lab rats? Using reward schedules to drive engagement
• Do you ever say, “MySpace is sooo ugly?” This blog’s for you…
• Does Facebook reflect your true friendships? How about e-mail?
• Facebook Apps: Why they’re focused on fun instead of utility
• Is your website a leaky bucket? 4 scenarios for user retention
• Public and private spaces, and why YouTube comments are so awful
• Social gaming design – Bartle types versus Web 2.0 participation pyramid
• Social network death spiral: How Metcalfe’s Law can work against you
• Technology always changes, but people always stay the same
• The design of social spaces
• User retention: Why depending on notification-driven retention sucks
• Users, customers, or audience – what do you call the people that visit your site?
• Why your friends list gets polluted over time
• Your site will succeed or fail in the first 10 seconds

Online advertising and social network monetization

Social web product have unique characteristics as it applies to online advertising. These posts cover some of the issues around key topics such as CPM rates, behavioral data, ad revenue modeling, etc.

• 5 factors that determine your advertising CPM rates
• 5 things that make your social network monetize like crap
• 7 ideas for billion dollar startups in online advertising
• Data portability: Is the social network data you’re hoarding treasure or trash?
• Counting your big pile of Benjamins: 5 startup tips for maximizing ad revenue
• How NOT to calculate ad revenue
• Online advertising during a recession: 5 key trends for ad-based startups
• Vertical ad networks: What are they, and why did Cox just buy Adify for $300MM?
• What’s the value of a user on your site? Why it’s hard to calculate lifetime value for social network audiences
• Your ad-supported Web 2.0 site is actually a B2B enterprise in disguise
• Super Rewards and the leadgen side of Facebook virtual currency – can it last?
• Remnant ads and the advertisers who love them
• Ad targeting talk from Community Next: People Not Pages (updated x2)
• Revenue, ARPU, Funnels, and RPM: My talk from Startonomics on Revenue metrics

Metrics

Without metrics, web entrepreneurs are just flying blind. These essays cover some of the organization and development issues around instituting a metrics system – what to measure, in what order, and how to implement them.

• 5 steps towards building a metrics-driven business
• The first 6 steps to homegrowing basic startup analytics
• Are you misusing Alexa numbers? (Probably)
• Lessons from the casino industry on engagement metrics and lifetime value
• Obama and McCain: How political marketing has evolved from offline to online
• omg I’m just a startup, I can’t do those fancy analytics!
• Recency Frequency and Monetization (RFM): Optimizing your notifications strategy
• How to measure if users love your product using cohorts and revisit rates

Media and games

Traditional media, including TV, music, games, and movies are at a crossroads. Here are some thoughts on how the industry is changing and evolving.

• From analog dollars to digital pennies: The crisis in traditional media
• Game design tutorial at the GDC
• MySpace versus Facebook: Analysis of both traffic and ad revenue, using Google Trends
• What every Web 2.0 entrepreneur should know about virtual goods
• Social Gaming Summit: Recap and observations
• Early adopters vs the Mainstream: Google Insights points out websites only used by Silicon Valley nerds
• YouTube vs Webkinz: Case studies for new product adoption
• Prosper.com and peer-to-peer lending in the economic downturn

Entrepreneurship and startup life in San Francisco

Just a couple thoughts on things I’ve encountered while arriving in SF.

• 10 tips for meeting people at industry events
• 5 ways to break past the San Francisco echo-chamber
• Are Web 2.0 startups wasting their time with Web 2.0 early adopters?
• Couple quotes on Facebook in Wired, Fortune, and NYT
• How do you find a badass co-founder, Part 2
• How do you find a badass co-founder?
• How do start a professional blog: 10 tips for new bloggers
• Is blogging worth it? What’s the ROI?
• Moving to SF and joining the tech community – Lessons from my first year
• What’s an Entrepreneur-in-Residence?
• What is your W2SAT* score? (*Web 2.0 Startup Aptitude Test)

As always, here are some of the articles/URLs I’ve been tweeting lately.

If you want to grab these in real time, you can follow me on Twitter at @andrew_chen.

• ever heard of the ultimatum game from behavioral economics? Made me think of that when reading this article: http://bit.ly/GDQX
• zomg, shiba inu puppy cam! http://tinyurl.com/3nwgde
• miley cyrus on pace to become billionaire by age 18: http://www.msnbc.msn.com/id/23051344/
• true or false: economists > political scientists > sociologists? FWIW, gre scores for all of them: http://bit.ly/2tVhEU
• New malcolm gladwell article, Can underprivileged outsiders have an advantage? http://bit.ly/4395X0
• wow, how skimmers+videocams are used to steal credit card numbers and PINs from ATMs: http://tinyurl.com/tb8hj
• wow, extremely detailed graphs and data on virtual currency exchange rates: http://www.gameusd.com/news0730.htm
• wow, zynga is dominating social gaming on myspace: http://tinyurl.com/56wj6v
• Lol, it’s kinda relaxing! Scientific Attempt To Create Most Annoying Song Ever: http://tinyurl.com/55bl9d
• good to see big brands embracing good ol UGC: http://tinyurl.com/5czoeq
• fun data in the form of a youtube video: http://tinyurl.com/2rzrn9
• neat lil NYT article about my hometown: http://travel.nytimes.com/2008/10/31/travel/escapes/31american.html
• rt @venturehacks Why startup pitches fail. See http://tinyurl.com/6prpes

Usability notes from iPhone apps
I found these interesting notes on how people use the iPhone, mostly usability-oriented, focused on a couple key applications and areas. It’s useful info for anyone who is building on the iPhone platform, which has some very unique interactions that even expert users have a tough time mastering.

The presentation covers the following apps:

• browsing
• time/calendar/alarm
• maps
• youtube
• downloaded apps: air hockey, koi pond, labyrinth
• iphone store (including pricing, reviews, etc.)

Then there’s a nice wrap-up of best practices for iPhone app usability, which is always nice.

Here’s the full presentation – you’ll have to click through the link if you are reading this in an RSS reader:

Poll results are in!
I recently looked at the poll I put up about blog topics – very interesting.

It looks like people are absolutely more interested in analytics and metrics, particularly in user engagement, and much less interested in social gaming, social networks in general, etc. I assume that it’s because for the social stuff, there’s a lot of specific coverage on that in tech blogs like Techcrunch and the like.

I haven’t seen as much stuff out there digging into analytics around engagement/viral in social applications – most of the good analytics out there focuses on e-commerce, etc., so I imagine that the appetite is out there for a more focused analytics conversation.

No love for social gaming
Most interesting to me was that there’s very little interest in social-gaming – it was tied at the bottom with B2B and vertical social networks. I figured that given social gaming’s hotness in the investment (and social networking sector), that it would rank quite high. Good thing I ran this poll!

Anyway, here are all the results below:

	Answer Text	Votes	%
	Analytics and metrics	69	20%
	User retention/engagement	68	19%
	Viral marketing	55	16%
	Online advertising	39	11%
	Product management	39	11%
	Social network platforms	30	9%
	B2B and vertical social networks	23	7%
	Social gaming	23	7%
	Other…	5	1%
	TOTAL	351

Click below for the full screen view of Facebook with huge ads

If Facebook is seeking a revenue model, let’s hope it’s not ads
There’s been a couple posts lately about Facebook and its revenue model, particularly this article on Venturebeat, written by Eric Eldon: In bid for international money, Facebook takes gifts off dollar standard. (There was also an earlier article on Techcrunch: Facebook May Be Growing Too Fast.) In either case, it’s absolutely indisputable that Facebook eventually needs to have a business model to be successful long-term.

Now up to this point, the company has tried a number of very social-network specific advertising such as:

• Facebook Pages
• Social ads
• Sponsored groups
• … and other ad solutions listed here.

These are all great, innovative products. I think it’s natural that anyone at a hot new company would primarily focus on the outputs that it could uniquely produce. That’s at the heart of differentiation and defensibility in the market

On the other hand, we have to consider what advertisers want, and ultimately, all these social campaigns at Facebook just aren’t the mainstream. Instead, this social media advertising gets grouped into “experimental budget” by ad agencies, which really control the purse strings in the brand advertising world.

What is experimental ad budget?
Typically, an ad agency is spending the bulk of its clients’ dollars on media they know performs. It used to be all TV, but we’re also seeing a lot of spend in search, buying premium spots on brand publishers’ homepages, etc. We’re often talking about 90%+ spend in these mainstream categories, and then a much smaller amount in experimental advertising.

Jeremy Liew recently referenced a WSJ article on experimental advertising which states that this budget is getting slashed as the recession develops:

In recent years, marketers have set aside a portion of their ad budgets to experiment with digital technologies such as Web video, mobile phones, gaming and virtual worlds. But with broader economic turmoil reaching Madison Avenue, these “experimental” budgets are among the first to hit the cutting-room floor.

Chrysler LLC has already slashed its experimental ad buys. With each ad dollar facing additional scrutiny, especially in the hard-hit auto industry, these ad buys will now make up about 5% of the auto maker’s marketing budget, down from as much as 10% in previous years, says Deborah Meyer, Chrysler’s chief marketing officer.

In good times, the maker of Chrysler, Dodge and Jeep brands tapped technologies like gaming and mobile to build awareness of its vehicles. “We won’t experiment in a lot of things that are fun to have. All of our dollars have to go to hitting in-market shoppers with the appropriate media,” Ms. Meyer says.

Areas like mobile, virtual worlds and widgets are expected to be hit particularly hard, as it remains unclear what kind of impact ads in these media have. These campaigns often reach a small number of people, and standard measurement systems have yet to be developed. “When we get into the need to drive results, you can’t spend money on the experiments and hope to keep your job and get your sales goals,” says Peter Kim, senior partner at Dachis, which advises marketers such as Philips Electronics NV’s Philips Healthcare and Johnson & Johnson on marketing strategies.

Jeremy also has two other relevant articles on this topic, Which online media companies will survive the ad recession? and New forms of advertising are hard. Both are worth reading.

Well, what do advertisers want?
The point is, advertisers are used to buying something very specific – they want IAB standard ad unit sizes, they’d like to do homepage takeovers and roadblocks, etc. And of course these are units that Facebook does not emphasize in their ad revenue process, because they are often annoying and obtrusive. Again, let’s take a look at ths mockup with the Facebook homepage covered with ads for the next James Bond movie.

Annoying, right?

I hope as an avid Facebook user that it never goes this far, yet as an online ad guy, I fear that’s what it’ll take to truly unlock the revenue behind Facebook’s pages, at least a long as it takes to convert the rather risk-averse agency community into building social media into their main budgets.

Given the frequent comparisons between Google and Facebook, it’s also educational to see how long it’s taken for the Mountain View giant to conquer the hearts and minds of Madison Avenue. Quite frankly, they haven’t been successful yet. That’s why after years of very aggressive talk about commoditizing the agency business, and making the entire ad buying process more efficient, you’re seeing Google account reps buying candy and otherwise sucking up to agencies. The article states:

Advertisers are grappling with the idea of Google, which spent many of its early years avoiding — and infuriating — advertising agencies, now shifting to embrace them. […]

“We understand that maybe we haven’t been the best partner over the years,” said Erin Clift, the director of agency relations at Google.

Google could avoid ad agencies when it sold only search advertising, where it is dominant. But now that it has a wider set of products in more areas — including social media and virtual reality — it finds that it must work harder to drum up business, particularly because of the lingering hard feelings.

That’s right, 100B in market cap and they have to resort to bribing agency folks with candy! That is what is in store for Facebook’s efforts in trying to get brand advertising agencies to get excited about them.

Quick thought on an impending cultural revolution
I also want to point out that if this is what is in Facebook’s future, their culture is bound to look more like Yahoo than it is to look like Google. This is because ultimately, it means that technology will have very little to do with the success of monetization on Facebook, and instead it’ll primarily be centered around sales and marketing execution. These are the hallmarks of a media company that derives their primary revenue stream from brand ad agencies.

In fact, more complex technology and coining terms like “social graph” will only hurt, not help, in realizing the monetization potential around Facebook. What this means, unfortunately, is that there may be a subtle transition of power and emphasis from the super smart geeks who built Facebook and more into the hands of polished ad sales staff who can make money.

Every technology firm engaged in the ad network industry goes through this transition. It was true for the team at Revenue Science, and I’ve shared many conversations with Bay Area-based ad networks who figured out the key to their success was not algorithms, but happy hours with clients.

When will we know when Facebook starts prioritizing ad revenue?
OK, now all of this said, I think I’m making these predictions all too soon in Facebook’s trajectory. The fact is, Facebook is really not acting like they care much about ad revenues.

Here’s the checklist of potential actions that would tip off an intent in that direction:

1. Creating content categories or building out public spaces with focused content, which will give their ads more contextual relevance than just profiles (like a public blogging platform on top of Facebook)
2. Decreasing their international traffic, which is tough to monetize and costs just as much money, and/or de-emphasizing the amount of international traffic from their marketing
3. Allowing more standardized ad units, particularly high-end content like video, homepage takeovers, etc.
4. Taking strategic investment or otherwise strongly allying with a giant media company to take advantage of their brand advertising teams (Viacom?). This is to counter MySpace’s Newscorp resources
5. Emphasizing a specific audience for Facebook, at least to ad agencies and marketers, rather than trying to stay horizontal and representing their traffic as a giant gray neutral blob
6. Creating more opportunies for search, and/or integrating content or creating partnerships which try to increase the number of commercial searches on the site
7. Acquiring an ad network, not for the technology, but for the people – and appointing a brand advertising guy as Chief Revenue Officer
8. Stopping their clearly public Google envy :) and starting to think more like a publisher rather than a technology platform – perhaps this will be exhibited by less Google hires, or a very big splashy media hire, etc.

Anyway, those are some wild ass guesses! But if any of those happen you saw it here first.

Like this post? Get this blog by RSS or email here. Also, check out a ton of other essays on related topics here.

Please take a second to vote below! Also, suggestions on stuff outside of the checkmarks are appreciated.

Click this link if you’re not seeing it.

Thank you!

Well, it turns out that Star Wars fanatics are easier to monetize, but are harder to find at scale!

More recession talk = More interest in direct monetization
There’s always discussion about the weaknesses of ad-based revenue models for consumer internet, and with recession chatter increasing every day, many companies are turning to freemium based models. Silicon Alley Insider recently wrote about this in an article called Revenue Crisis: Here Come The Pro Accounts. And of course, there is constantly discussion around the example of 37 Signals, who are notable critics of business models that give away product for free, described by the the Techcrunch article 37 Signals Drives Another Company To The Deadpool. Here’s another from Slate called A Radical Business Plan for Facebook: Charge people.

Direct monetization models versus indirect monetization
To restate the general argument in my own terms, there are basically two kinds of companies on the internet (and btw, I’ve discussed these two groupings of companies at my talk at Startonomics – you can check out the video here and the slides here):

• Direct monetization, aka Advertisers: Direct monetizers charge money for their products, via subscription, ecommerce, virtual items, etc. They typically have a small, focused group of customers.
• Indirect monetization, aka Publishers: Indirect monetizers don’t charge money to use their product and in fact, often give their product away. They chop their audiences into pieces (using content to differentitate between them) and sell the targeted audiences to companies who directly monetize them.

Now note that Direct monetizers and Indirect monetizers have very different problems:

• Direct monetization: The biggest issue is cost per acquisition and limited size of their customer base.
• Indirect monetization: The biggest issue is zero cost user acquisition and identifying user intent (via targeting)

And of course, the central issue is that direct monetization has the huge advantage that you can “make money as you go” and maintain a profitable trajectory at every point. This is great for the bootstrapped startup.

Compare this to the indirect monetization companies which often need to reach a very large critical mass, burning lots of capital along the way, until it gets large enough to sell their audience segments in large enough chunks to be interesting to advertisers.

From the venture capitalist’s point of view, the indirect monetization models are often able to produce larger exits, and thus the “go big” mantra holds here. The reason is that the indirect monetization methods are often appropriate for companies who have extremely horizontal audiences (like search, email, video, etc.) combined with viral growth.

From the VC’s perspective, the direct monetization models can often be less desirable because they may have a small customer base that can only be reached by expensive marketing techniques. As a result, there ends up being a smaller exit because the startup exhausts all their marketing channels to reach their customer base and may not be able to grow beyond that. This is why, although it may be highly profitable to build software for orthidontist offices, these companies often end up lifestyle businesses and not venture-returns businesses.

Which one are you?
I want to stop here and ask, which method best suits your company? I’ve found that there are several Web 2.0 companies floating out there that should probably charge for their product because they are niche products, but instead they are opting to go free and ad-supported. This may be a mistake. Similarly, there are products out there with wide audience appeal that may generate more revenues as indirect monetization models (for example, much of digital content).

Both strategies work: Compare MySpace versus World of Warcraft
Ultimately, this is an optimization of two variables simultaneously. One variable is the size of the audience, and the other is the revenue potential.

The tension is that:

• Size of audience is determined by breadth of appeal
• Revenue potential is driven on intent and passion of audience

Oftentimes, of course, these two variables are at odds. It’s the rare product that EVERYONE will pay for, and often you have to choose between monetizing a small group of fanatics at high rates, or monetizing a huge group of casual users at low rates.

The two corners of this model, of course, are MySpace and World of Warcraft.

• MySpace is an amazing indirect monetizer – they have a HUGE audience, but but make very little money off of each user. Even with 10s of millions of uniques, they make cents per user per month. But this all adds up to nearly $1B in revenue per year. Note that the first version of MySpace was relatively cheap to build (hey, it’s just a website!)
• World of Warcraft is an amazing direct monetizer – they have a much smaller audience (<15M subscribers) but make a ton of money off of each user. They charge a $15/month, and even with their much smaller audience they make nearly $1B in revenue per year. Note that the first version of WoW was quite expensive (into the 10s of millions!)

The point is that both strategies work – the question is, what can you learn from each of these to figure out which model works best for you?

Pretty interesting data:

SAI publishes a post called Ad Rates Dropped 11% In The Third Quarter.

I think this tells you that if you have a direct monetization method for the audience (and context) within social networks, music, and entertainment, this is a great opportunity to pick up users on the cheap. Companies that are selling subscriptions to DVDs, music, etc. will all benefit from this. Games supported by virtual goods and subscription should stand to benefit as well!

Leadgen blog
For those of you guys who are interested in Offerpal and Super Rewards, and want to learn more about the leadgen industry, I couldn’t recommend Jay Weintraub’s blog more.

Here are a couple of my favorite articles:

• The Mobile Subscription Services (aka Ringtone) Ecosystem
• Incentivized Marketing – An Updated Look
• New Strategies in Incentivized Marketing (B2C)
• Lead Generation – Quality over Quantity
• State of Remnant Advertising
• Direct Response In A Behavioral Marketing World

Jay also doesn’t write nearly enough, but perhaps this will send a torrent of traffic his way that will encourage him to spend more time blogging :)

Also, a great preso
Most recently, Jay published an amazing presentation “You Give Leads A Bad Name” on leadgen dynamics including topics like:

• history of leadgen
• ad creative
• user flows
• mobile services
• leadgen
• scammy tactics
• … and finally, the evolution of leadgen into more sophisticated sites (that give users value)

Watch for the slides on Mint.com and Billshrink.com, in particular, which are valley darlings (and are mentioned very favorably).

You should probably watch in full screen!

You Give Leads A Bad Name

Quantitative marketing in politics
There was recently an article in Slate on the topic of the effective of robo-calling versus text messages, and discusses in some detail the ways that A/B testing is used in the political world. I previously wrote about a similar topic in my blog post Obama and McCain: How political marketing has evolved from offline to online.

Why look at political marketing strategies?
I find the world of political marketing very interesting, as it requires marketers to maneuver without significantly their product (their candidate), but rather only changing the messaging and story. It’s a pure marketing pitch. Compare this to tech marketers who often depend on new features to compete. It’s sort of like asking the question:

How would you sell bottled water?

That is, given a completely commodity product like water, companies like Evian, Fiji, Voss, Arrowhead, and others manage to sell a product using a fundamental pillar of branding: STORY.

I find politics interesting for the same reason, because it’s all about storytelling and narrative. I wanted to point everyone to a recent story in the New York Times on the difficulties in establishing a narrative for McCain, in the article titled The Making and Remaking of McCain. Worth reading.

Robo-calling in the Slate article
Now the article in Slate had some interesting passages that I wanted to highlight, on cost per acquisition, split testing to encourage voter turnout, and other fundamental ideas that every quantitative marketer should have in their toolbox.

On the cost of robo-calling:

Robo-calls are the pyrotechnics of politics: They create a big disturbance, but they don’t have a prolonged effect. Numerous studies of robo-call campaigns show that they’re ineffective both as tools of mobilization and persuasion—they don’t convince voters to go to the polls (or to stay away), and they don’t change people’s minds about which way to vote. So why do campaigns run robo-calls? Because they’re cheap and easy. Telemarketing firms charge politicians between 2 and 5 cents per completed robo-call; that’s as low as $20,000 to reach 1 million voters right in their homes.

On using split testing to test political marketing strategies:

[…] Text messaging is different: We pay attention to short messages that pop up on our phones. These conclusions arise out of work by Donald Green and Alan Gerber, two political scientists at Yale whose book, Get Out the Vote: How To Increase Voter Turnout, is considered the bible of voter mobilization efforts. Green and Gerber are the product of a wave of empiricism that has washed over political science during the past decade. Rather than merely theorizing about how campaigns might get people to vote, Green, Gerber, and their colleagues favor randomized field experiments to test how different techniques work during real elections. Their method has much in common with double-blind pharmaceutical studies: With the cooperation of political campaigns (often at the state and local level), researchers randomly divide voters into two categories, a treatment group and a control group. They subject the treatment group to a given tactic—robo-calls, e-mail, direct mail, door-to-door canvassing, etc. Then they use statistical analysis to determine whether voters in the treatment group behaved differently from voters in the control group.

On the analysis of “funnels” in converting contacted people into voting:

Having campaign volunteers visit voters door-to-door is the “gold standard” of voter mobilization efforts, Green and Gerber write. On average, the tactic produces one vote for every 14 people contacted. The next-most-effective way to reach voters is to have live, human volunteers call them on the phone to chat: This tactic produces one new vote for every 38 people contacted. Other efforts are nearly worthless. Paying human telemarketers to call voters produces one vote for every 180 people contacted. Sending people nonpartisan get-out-the-vote mailers will yield one vote per 200 contacts. (A partisan mailer is even less effective.)

Anyway, I won’t quote the entire article, but I personally found it pretty fascinating.

Underlying metrics
Anyway, the coolest part to me in all of this is the interplay between the soft stuff and the hard stuff. The soft stuff is the candidate, including brand, the speeches, the fashion, the messages, etc. The hard stuff is figuring out, once you have the candidate that you’re meant to market, how to break everything down into Dollar per Vote (which sounds kind of sinister when you put it tha way). Or perhaps, you could even calculate it down to Dollar per Electoral Vote.

Really this is a measure of the efficiency in which a well-run campaign is able to put the donations raised into the raw results tha they want.

Anyway, if anyone’s read other interesting perspectives on quantitative marketing in the political realm, I’d be interested in hearing about it.

Here are all the links I’ve twittered over the last week – If you want to grab these in real time, you can follow me on Twitter at @andrew_chen.

• inside the creative process at The Onion, it takes 600 headlines to generate 18 good ones: http://bit.ly/3GzXKZ
• twitter analytics, neat! I twitter the most mornings early in the week (mondays especially): http://xefer.com/twitter/andrew_chen
• pretty cute flash game: http://tinyurl.com/4uee65
• distribution of income versus distribution of taxes: http://bit.ly/1xLxT5
• lol, i’m using qwitter to figure out when people unfollow me, turns out it’s when i write a personal status message! http://useqwitter.com/
• dino run really is a great flash game: http://www.pixeljam.com/dinorun/
• luxury consumption index in free-fall: http://tinyurl.com/59a9jp
• cultural differences between games and web people, from adam martin (ex-NCsoft) http://bit.ly/16Sm4S
• venturehacks on lean startups: http://tinyurl.com/5bfr6d
• leveraged sellout is much less funny now that bankers don’t seem so baller anymore: http://www.leveragedsellout.com
• crowdsourcing sex: http://tinyurl.com/6d5ggu
• the facebook platform is dead! declares @rafer: http://tinyurl.com/6oqk54

A/B testing is fun ;-)
Generating candidates for A/B testing can be a great source of entertainment for your team. It’s easy and fun to generate dozens of potential candidates for a new headline, subtitle, picture, or other element of an important page. It’s also a great exercise in both the qualitative, consumer psychology skillsets required, as well as the quantitative set.

I’ve gather a couple rules of thumb in helping you generate good candidates for A/B testing below:

• Brainstorm the RIGHT way
• Dive down into potential customer motivation
• Go for high variariance approaches
• Test big things first, smaller things later

Let’s dive into each of these…

Brainstorm the RIGHT way
First off, not all brainstorming is created equal – you want to make sure you are going for lots of quantity, that the most senior person in the room doesn’t “run” the whiteboard, and a bunch of other guidelines that you can find in this article on IDEO’s brainstorming techniques. I generally find that after brainstorming individually on dozens of candidates, you can build on very interesting themes and start to coalesce the entire process.

Dive down into potential customer motivation
One important issue is that every product and every page within your product likely caters to multiple needs. Influence, the classic book on persuasion by Robert Cialdini, enumerates many of them. Is it:

• Reciprocation
• Commitment and consistency
• Social proof
• Liking
• Authority
• Scarcity
• etc?

Or alternatively, you may have specific ideas about value propositions or user emotions – for example, a social network like MySpace could be marketed around:

• Customizing profiles
• Socializing and friends
• Media content
• Photos and blogs
• etc.

Who knows which feature is king? The point is, each one of these potential customer motivations and values probably deserves at least one, if not several, test candidates in your A/B test. The fundamental emotions driving your product have a huge likelihood chance of altering the outcomes of your split tests.

Go for high variariance approaches
Similarly, life is too short for the safe stuff. Because of the fact that you throw away all the bad candidates and keep the good ones, it’s in your best interest to try to make the good ones as good as possible! As a result, make sure you try to go for extremely polarizing, high-variance approaches.

For example, make sure you try candidates that:

• … are aggressive and in your face
• … use different graphical elements like videos versus text versus audio
• … are varied in length, like very very long or very short
• … may offend certain subsets of your audience
• … are commanding and direct the user

Typically, in an A/B test I will usually have a control, then a candidate that incrementally improves on the control, and then a couple candidates from left-field. As you try out more candidates and learn from the process, then often times you will start going with more incremental stuff to finish your optimization.

But early on in your experimentation process, remember to go wild!

Test big things first, smaller things later
Similarly, make sure that you prioritize the your tests so that you aren’t testing subtitles and paragraph copy when you could be trying out even more extreme stuff. Things like the user flow, the layout, “hero shots,” and other factors are usually much more important than smaller things like icons or specific sub-labels for forms.

As a result, oftentimes the best thing to do is to rush out some forms to test, then make things prettier and more finalized from there.

Anyway, I hope this was helpful, and if you have more ideas, please comment and suggest more ideas!

Eric Ries recently wrote a great blog called Three decisions to make on virtual goods, detailing the major issues IMVU had to deal with during its growth:

• User-generated content (UGC) or first-party content?
• Subscription or a la carte payments?
• Merchandising or gameplay?

The first question is particularly interesting, and Charles Hudson (formerly of Gaia, now at Serious Business) exchanged a couple emails with me on the topic.

He gave me permission to share his email below – note that this convo happened in December of 2007, so it’s a bit dated, but I think it’s still quite applicable:

from    Charles Edward Hudson
to    Andrew Chen
date    Tue, Dec 11, 2007 at 9:55 AM
subject    Re: question on virtual goods

The question you bring up is a big one and it’s really hard for me to be succinct (but I’ll try). There are a ton of advantages and disadvantages to each approach. I’ll tell you my thoughts (others at Gaia might feel differently) on why the user-generated model is riskier but potentially a bigger opportunity if you can reach scale. Below are my quick thoughts on the 3 biggest drawbacks to a UGC/DIY model as a starting point:

-You need to have users actually create stuff to make the UGC model work (DIY vs DIFM) – If you decide to go in the “do-it-yourself” model from day one (as opposed to “do-it-for-me”), you’re making a really big bet about the type of users you’ll attract. You’re going to need power users and creators who build lots of stuff to make your world or community feel vibrant. Instead of being constrained by your own ability to generate assets, you’re constrained by the creative cycles of your user base. I’m of the opinion that it’s actually better to assume the DIFM use case first and then slowly offer users DIY activities once you’ve figured out what it is that they want to customize or do within your environment.

-You need to provide users with the tools to actually create and manage those assets if you go the UGC/DIY route – You have to make a meaningful investment in tools (or at least expose the tools and systems you have) very early on if you want users to create stuff for you. And the type of tools you expose will dictate who builds. If you make really simple tools, you’ll get wide participation. Expert tools will likely narrow who produces content for you. I’d rather work on building a really great end-user experience than building great tools, but that’s just me.

-You have a lot less control over the world and economy when you don’t control the process of creation – To your point, it’s really hard to manage an economy or any system if you can’t control the inputs. It’s not so much about monopoly pricing as it is about being able to manage the economy – price controls, inflation, etc. You also don’t have to deal with all of the property rights issues and user-to-user copyright issues that emerge when you allow users to create and then resell their own goods.

I think the real challenge for someone like Second Life is finding enough people to create the world to the point where non-creators can simply join and have a great experience. I think that’s why they’re having a hard time growing. If you look at their community, they have a fairly small but really dedicated group of people. Those people are bearing a pretty serious creative burden to get the world and the experience to the point where it’s useful to and usable by a wider variety of people.

At the end of the day, I’m of the mind that the best way to become a platform is to build a great application. If you’re successful, other people will want to build on top of you and you’ll become a platform whether you want to (a la Facebook) or not (a la MySpace). Setting out to be a platform from day one (and that’s what I think Linden and others have done) is just a much harder road. But it you make it work, you avoid all of the retrofit problems that happen when applications need to rework themselves to be platforms.

So to summarize, according to Charles you face a number of issues:

• You need more creative users in your world, to author the UGC
• (Related to above, you need to lower the bar for participation in creation)
• You need to build an ecosystem of tools to help the authors
• You need to accept that it’s harder to manage the economy – pricing, inflation, etc.

Anyway, I found these points quite insightful – they should be useful for anyone looking to chose between first party and UGC virtual items.

Found on YouTube: Mario and Luigi’s insightful commentary on MySpace top friends

Cultural differences are always interesting!
I got interested in the games world first as a consumer of video games, but after I worked on an unsuccessful project to monetize MySpace using ads, I got interested in the monetization potential of virtual items in social products. For the last 2 years, I’ve been wandering around on the edges of the games industry to try to cross-pollinate some of the best ideas with what I knew from the web world.

Early on, after attending the Game Developer Conference and speaking with folks from many of the top publishers and studios, it became clear that there were lots of interesting cultural differences between web folks and games folks. I wrote some of these points down a while back and I thought I’d share them.

I want to caveat that these are purely anecdotal and my own experiences, and I’m sure that I’m overgeneralizing ;-) I also think that people that come from the casual games world (and in particular flash games) are much more similar to web entrepreneurs – the aliens I talk about are mostly big packaged games people. So please share your opinions in the comments if you disagree or have another perspective.

But here are the major ones:

1. Eyeball worship vs. Game genre worship
2. Distribution vs. Content
3. Utility vs. Experience
4. Open vs. Content gating

Let’s drill into each of these…

Eyeball worship vs. Game genre worship
First off, one of the big surprises for me was that many of the folks working at big games companies like EA have a very specific type of game they want to work on. Many of the folks I talked to wanted to make so-called “hardcore games” – very rich, deep, FPS/RTS/RPG/etc packaged games that sell at Walmart, and were completely uninterested in anything else.

While I excited about building simple Web-distributed games that could be played by millions of people, for many of these folks, if it didn’t look like a game, didn’t have monsters and guns, it was uninteresting. In fact, there was a pretty derisive view of folks who make so-called casual games as lower in the food chain.

This reminds me of a project I worked on a long time ago in the video space, pre-YouTube. I had interviewed a bunch of art students at Unviersity of Washington to talk to them about publishing their videos online, and they were very uninterested. For these art students, they had such a romantic sense of what it would be like to show your work in a theater, at Cannes, that the idea of millions of people watching a 400×415 pixel player seemed completely uninteresting. Perhaps the hardcore games folks I talked to felt the same way about their work.

The analogous concept in the web world is probably that a lot of entrepreneurs only want to work on “cool” startups involving fancy technology. They are less likely to think along the edges for products targeted at different (possibly more mainstream demographics). I also think that web folks get more excited about the eyeballs factor than anything else. The more simple, stupid, and widely used something is, the better!

Distribution vs. Content
Another interesting difference was the perspectives around content. For many of the games people I met, the content is everything. How good your game is perceived to dictate its ultimate success. I think this makes sense in an industry where distribution is essentially commoditized! The big publishers have many of the same relationships, and games developers in general have been outsourcing their distribution expertise out to the publishers for the past couple decades. As a result, it seems clear that the only place to compete is in the content of the game, rather than in the distribution.

Compare this to the web entrepreneurs who have to deal with the constantly changing landscape of distribution. Many of the top Facebook apps were simpler, dumber, and better distributed than their competition, and distribution in itself can be a competitive advantage. Eric Ries recently wrote about the distribution techniques that have recently been found for the iPhone App Store – these techniques include a primitive version of SEO via the App Store search function, as well as folks who constantly release updates to their app to try to get on the New and Hot list.

And of course, ad networks, affiliates, and leadgen companies represent the logical extreme in the distribution equation. Because they are selling other peoples’ products, they focus exclusively on distribution and differentitation via novel techniques and analytics.

It’s clear that both communities have a lot to learn from each other on this one, but because of the fact that distribution is extraordinarily important in the new social network ecosystem, I think this is why we’ve seen the top games coming from Web teams rather than Games teams. (With the possible exception of Playfish!)

Utility vs. Storytelling experience
One of my favorite cultural differences is the way web folks think about the role of their products in peoples’ lives. There’s often talk about making your product as “useful” as possible, or “social utility.” In the world of utility, oftentimes the main factors that are discussed involve terms like:

• pain points
• efficiency
• productivity
• ROI
• maximizing
• etc

These terms are great, and the world is better off for having products that make us all better worker bees!

Compare this to many games discussions, like the ones I sat through at GDC, which involved concepts like:

• characters
• plots and storytelling
• mood
• music
• fun
• etc

Now, I think that the productivity-inclined have their claim to the world, as does the fun/entertainment games people. But the intersection of this, in web media, is where the fun happens. For example, is the fact that Facebook has such an efficient newsfeed system a good thing, or a bad thing? I think it depends on whether or not you feel like the process of exploring peoples’ profiles and clicking through things as a good thing or not? In the MySpace world, given the degree of customization, you might argue that it’s more game-like in the way that it encourages people to click around and explore, whereas Facebook is clearly more efficiency-oriented.

Both approaches have their advantages, of course – and there are times where I use Facebook as a utility and times when I’m using it for time-wasting. The tradeoff between the two approaches are definitely interested to think about as your product is being constructed.

Open vs. Content gating
Related to the efficiency versus experience distinction, web products are very likely to make things very open and give the users all the features upfront. It’s very rare that you constrain what the user can do, and as a result, there’s no concept of leveling or grinding. As a result, oftentimes the experience that you get at the beginning is the same as the experience you have later on.

Games, on the other hand, have a clear concept of advancement and otherwise “content gating” their users. By withholding levels, powerups, weapons, trophies, etc., it creates motivation from the user to keep on playing. They say, “just… one… more… game…!!”

The Wikipedia article on this is instructive:

The most common form of level treadmill is the practice of killing monsters for experience points. The player constantly chases after the next level in order to be able to defeat the next slightly stronger monster. The outcome of MMORPG combat tends to depend more on the character’s numerical statistics than the player’s skill. Thus there is usually little for a player to do beyond clicking an attack button until he or she wins, or is forced to flee when nearing death. So whether fighting small rats or large demons, the player is performing essentially the same actions, the only difference being the larger numbers in his or her character and the monster’s attributes. In the eyes of players, the player is essentially running forward while going nowhere, as on an exercise treadmill.

As a result of this treadmill, there is a constant pressure for players to stay engaged and retained as customers. But the flipside of this is that it’s not enough to build one product – instead you build 70 product variations, and call each one a level!

Other observations?
I’d love to hear other thoughts on this issue, and any places where I’m overgeneralizing :) Comment away!

If you liked this article, check out my other essays here.

UPDATE: Adam Martin (formerly of NCsoft) writes up his views here, from the perspective of a games guy.

Turns out it’s harder than you think to build a vertical social network for Star Wars fans

Are you trying to build a vertical social product?
One of the most common questions I get on viral marketing is structured as the following:

I’m trying to build a social network targeted at X. How do I make successful with viral marketing?

Typically, the product in question will be some sort of subset of a general social network, such as:

• parents
• sales professionals
• fans of a TV show
• car lovers
• etc

and in fact, to generalize, a vertical social product might also include products like:

• a sci-fi themed MMORPG
• social products aimed at the Twitter audience
• products for a certain demographic
• people with webcams
• etc

Catch my drift? Well, let’s get even more specific about what it means to be vertical versus horizontal.

Horizontal versus vertical products
Basically, the definition how vertical or horizontal something is depends on what % of people are likely to use it in some form. The most horizontal products become core features of the internet, such as:

• browsers
• search
• email
• mobile
• video
• etc.

For this reason, products centered around communication or content are often considered very horizontal because a lot of people are likely to use them. These end up being the most viral products, for a number of reasons we’ll discuss later.

Compare this to more specific products or features, such as content on cars, or a sci-fi themed video game. These are things where if you were to select a random group of 100 internet users, you’d find many of them have no interest in using that site.

The place that becomes important is the “branching factor” of your viral invite strategy.

How vertical products drive down branching factor
In tree data structures, the branching factor is determined by the number of children that a node has. It can be represented as the variable “num_friends” in the following equation:

users * num_friends * conversion % = new users

If you think of a viral loop as generations upon generations of users inviting each other, it’s very important that each user has a big branching factor, thus inviting enough of their friends to propagate the viral process. (With the usual caveat that you must balance your viral incentives against your user retention goals!)

The problem with vertical products is that it automatically impedes the # of invitees that a typical inviter might reach out to. The % of those invitees that get excited about the product upon their first use of it might be quite low, and if they leave prematurely without inviting their group of friends, then your viral process may peter out. It adds an extra term here, % targeted, which can significantly drive down this process:

users * % targeted * num_friends * conversion % = new users

So as a result, the more vertical a product is, and the lower the % of invitees that may be interested, the harder it will become viral. And in a world of shrinking possibilities on Facebook/Open Social, overcoming a big quantitative hurdle.

But don’t worry, it might still be OK!
I’ll close out this blog with two thoughts on this subject, first on scenarios where vertical products can still work, and also situations where you don’t care about being completely viral.

Vertical products can still work because of the fact that some target audiences are very concentrated amongst each other. The business world is often like this, or teenage girls, also! If you build a product targeted at them, the networks are dense enough and interconnected enough that you’ll get some degree of virality that eventually peters out as you hit this network’s edges, but it works in the meantime. I think that Linkedin is a great example of a site that provides great value, and because peoples’ Outlook contact lists are populated with large target-rich lists of like-minded professionals, this can work quite well.

As a related note, products like Ning, mailing lists, Yahoo Groups, etc. also do an interesting variation of this where maybe you came in via a “for Moms” invitation but then you end up making your own “for pet fanatics” group. This basically converts vertical traffic into horizontal traffic, neat!

Similarly, it’s not required that a site become viral to succeed. Oftentimes, a more specific audience is able to generate key context which provides high advertising revenues and/or leadgen/CPA opportunities. Because of this, these sites can often buy significant amounts of traffic which are then amplified by some degree of viral marketing but is not completely driven by it. In this way, the viral marketing is just a “kicker” that drives down the cost per acquisition without pushing it down to zero.

Both cases can be successes – I’d encourage everyone to check out Dogster, Linkedin, IMVU, and Puzzle Pirates as three examples of vertical products that worked out their acquisition strategies.

If you liked this article, you can check out my related essays here.

I wanted to quickly give some linklove to Furqan Nazeeri, a former EIR from Softbank who’s been writing some great stuff on raising money. His blog is at Altgate.com, and the RSS feed is here.

He recently made a list of his popular posts which I’ll list below here:

Raising Capital:

• 5 Reasons Convertible Debt Sucks HOT
• 10 Tips On Negotiating With VCs NEW
• Dating…er…Fundraising Etiquette
• The Science & Art of Term Sheet Negotiation HOT
• How Liquidation Preferences Work HOT
• How Much Money Should I Raise? NEW
• How To Get Introduced to VCs
• How To Pitch To VCs
• Killer VC Pitch Deck NEW
• Top 10 Tips For Entrepreneurs Pitching VCs
• Top 10 VC Objections (And How To Overcome Them) NEW
• Triage NEW
• Venture Debt For Startups HOT
• What To Expect In Due Diligence HOT
• Why Dividends Matter HOT

Startups:

• ‘Tis The Season To Change The CEO
• 10 Things To Consider Before Joining a Startup
• 10 Web 2.0 Tips: $75 HOT
• All-Hands Team Meeting NEW
• Board Management Tips For Startup CEOs
• Customer Support 2.0
• Financial Modeling Tips For Early-Stage Startups NEW
• Hiring Your First Salesperson
• How To Demo Your Startup NEW
• How To Pitch The Press
• Interview with Professor Noam Wasserman
• Sales Learning Curve
• Startup Compensation HOT
• Startup Entertainment
• Strategies For Two-Sided Markets
• What To Include In A Board Package

I have been very lazy about updating this! Sorry!

If you want to grab these in real time, you can follow me on Twitter at @andrew_chen.

• myspace making $1B a year, 3x more than facebook: http://bit.ly/PJPc2
• wow, what a lineup. John Doerr, Ron Conway, Max Levchin, and more. RecessionBeat event coming up soon: http://bit.ly/790Ci
• email marketing still most effective channel by ROI: http://tinyurl.com/4xqg34
• btw, i blogged about prosper stuff (today in TC http://bit.ly/7c9Z3 (expand)) over a month ago! http://bit.ly/1ccDXG
• Recession-Plagued Nation Demands New Bubble To Invest In http://tinyurl.com/5jgfha
• Retweeting @adachen: OMG BOOMBOT 2 IS OUT!! http://tinyurl.com/3ola4n
• great article in the nyt on amazon vs ebay: http://tinyurl.com/4r3ovc
• peter thiel and pearlstein on charlie rose, via @pkedrosky: http://tinyurl.com/5xjnu8
• remixes of the iconic obama poster, featuring amy winehouse, clinton, luke skywalker: http://www.posterpage.ch/exhib/ex216oba/ex216oba.htm
• jean paul gaultier redesign for evian: http://tinyurl.com/5yemed
• watching the gf play fieldrunners on the iphone for hours and hours. Looks like this: http://tinyurl.com/46dkxq
• testing ppl for their moral philosophies on mechanical turk: http://tinyurl.com/4t924l
• ok that’s cute. augmented reality virtual panda pet: http://bit.ly/1pFSsU
• andrew trader from zynga references maplestory’s $30MM in revenue in 2007: http://bit.ly/10h9bJ
• WSJ article on cans of mackeral becoming the currency of prisons: http://online.wsj.com/article/SB122290720439096481.html
• 7 habits of highly ineffective terrorists: http://tinyurl.com/42wrbd
• incredibly fascinating – a list of unusual deaths over the last 100s of years: http://tinyurl.com/8t7oc
• damn it feels good to be a banksta, The Comic: http://tinyurl.com/4zhzwx
• VP debate in autotune with music and beat. UGC video at its best: http://tinyurl.com/49juz9
• P. Diddy + Kim Jong Il collabo: http://tinyurl.com/3vnxgg
• smokinggun page of the day – ohio woman busted wearing cow costume and chasing kids: http://snurl.com/403dp
• vitamin-enhanced, ginseng-fortified, peanut butter-flavored bottled for dogs: http://tinyurl.com/3pwual
• “Results 1 – 9 of about 9 for barrack obama” via @brit http://tinyurl.com/4jpylj
• compete traffic analysis on the personal finance space, seems like mint.com is doing well (via ad buying): http://tinyurl.com/5xftmz
• nyt article from 1999: “Fannie Mae Eases Credit To Aid Mortgage Lending” http://tinyurl.com/4ulb87
• getting into fancy nyc restaurants by bribing the maitre d’: http://tinyurl.com/6g3ddn

Blog vacation
Sorry for the blog vacation – I’ve been working hard on presentations for Startonomics and the Virtual Goods Summit, which has taken a lot of creative juice over the last 2 weeks!

Blogger’s block
Anyway, I’m suffering some blogger’s block right now. I’d love some suggestions on areas and topics you guys want to hear about. Click here to suggest something, or write me at voodoo [at] gmail. I’ll get back on the horse soon!

Today, I did a talk at Startonomics on revenue metrics, click to see the text summary. The conference was well put together and a lot of fun. I wanted to share the slides as well as the video of my talk below – enjoy!

A great new blog from IMVU cofounder
I recently have been reading a new blog by Eric Ries, one of the founders of IMVU. The URL is http://startuplessonslearned.blogspot.com and the RSS feed is here. I’ve found it to be a great source of thinking around both games and metrics, two of my favorite topics. It’s quickly become one of my favorite blogs.

For those of you guys who haven’t spent much time looking at IMVU, here’s a good summary from GigaOm, including the statistics below:

• About 1.7 million user-generated items have been uploaded to the IMVU catalog
• Users create most content offline with industry-standard modeling tools such as Maya and Blender
• Individual content creators can take their earnings and sell them on third party web sites
• As with Second Life, IMVU users retain intellectual property rights to content submitted to the IMVU catalog
• About 100,000 users are registered to be IMVU content creators, but the number of consistent creators is in the tens of thousands
• IMVU users often sell their credits to other users via third-party web sites — also fine with Cary. “We observe it sort of casually,” he said.
• Demographic: Largest cohort is teens; 60 percent of users are female, 60 percent are American
• 50-70K concurrency is typical, with peaks in the high 70s
• IMVU areas are more like virtual rooms restricted to 10 avatars or fewer; Cary estimated 90 percent of those room interactions are person-to-person, anyway.

Eric has already written some great articles from his experience at IMVU, including articles on:

• common distribution models for online businesses
• gaining distribution advantages on the iPhone
• doing market research for product feedback
• a/b testing products

I’d encourage you to read more games+web+metrics goodness.

KP beefing up on games
Another interesting note is that it looks like KP has recently been beefing up their games focus. These days, Eric is a venture advisor for Kleiner Perkins, after 4 years at IMVU. He also joins the recently made partner Bing Gordon, a cofounder of EA, who quickly went on a tear making investments in games companies like:

• Facebook social gaming powerhouse Zynga
• mobile games company Ngmoco (from EA veterans)
• GoGii (from the Jamdat founders)

They also have Chi-Hua Chien, who sourced the Facebook for Accel Partners, and who is obviously a great resource for the intersection of social web and games.

Anyway, I hope you enjoy the blog!

My friend Dan Cook over at Lost Garden put together some slides on productivity research which I thought I’d share below. Much of it references the typical “crunch” periods when delivering large traditional games:

Last week’s Twitter links

If you want to grab these in real time, you can follow me on Twitter at @andrew_chen.

Wow, there has been a ton of investment into games-related startups.

I hadn’t seen this table until today – I thought I’d share it. It shows the tremendous amount of investment that’s been thrown into the “new” games space, although I’ll note that some of the companies on this list are relatively mature. But lots of them were funded in the last year or so. Given that there are still tons of stealth companies out there, I bet that a comprehensive list would be >2x as large.

Read more about this on Jussi’s excellent games blog:

UPDATE: There was a set of new updates on Jussi’s blog, which I’ve added here.

I’m speaking at the Virtual Good Summit (plus a discount code)

I’ll be speaking at this year’s Virtual Goods Summit on October 10th, 2008 and I just wanted to share some details about the event. The Virtual Goods Summit 2008 is a one day conference focused on the emerging market opportunity for virtual goods and economies. This year’s conference will build on the success of last year’s event and dive even deeper into some of the key themes facing this emerging industry. The conference will feature a blend of panel discussions and expert-led breakout sessions covering everything from getting started with virtual goods to maximizing the revenue opportunity around virtual goods and virtual economies.

I’ll be giving a talk during one of the breakout sessions – Daniel James and I will be talking about how to apply a metrics-driven approach to analyzing and understanding a virtual goods business, using some of Daniel’s experiences at Whirled as an example. There will be a ton of great speakers and presenters at the conference – I’ll be joined by Amy Jo Kim (Shufflebrain), David King (Lil Green Patch), David Perry (Acclaim), Brian Balfour (Viximo), Sean Ryan (Meez), Nabeel Hyatt (Conduit Labs), and many other leading thinkers in this space.

I hope can you can make it out to the Virtual Goods Summit 2008! You can save 10% on the cost of registration by using the code “ANDREWCHEN” at checkout.

Conference Website: http://www.vgsummit2008.com 
Registration: http://vgsummit2008-andrewchen.eventbrite.com

Last week’s Twitter links

If you want to grab these in real time, you can follow me on Twitter at @andrew_chen.

As always, I can’t promise they are all work-related 

Quick intro to getting set up on analytics
I’ve been asked a few times recently, “Wow, these analytics you write about are great, but how does a startup begin to bite off the relevant parts?” This blog is to address these questions.

First, let me recommend reading a previous blog, called omg I’m just a startup, I can’t do those fancy metrics. In it, I cover some more general philosophical ideas about how to approach what to measure and what not to measure. Might be worth taking a look if it’s not too important.

Now let’s move on to the first couple topics:

Step 0: Pre-product
Initally, the product development process should likely be focused on big-picture qualitative information, like whether or not your business is addressing the right audience as well as the preferences for that audience. So don’t measure anything yet :)

Instead, spend your time gathering qualitative data, interviewing users, understanding the problem-behind-the-problem you’re trying to solve, and prototyping concepts.

Do this for a couple weeks!

Step 1: Prototypes
As you create prototypes of your product, you should throw up some free, simple analytics to get you some rough ideas of what’s happening inside the functionality. This likely means something like Google Analytics, although there is a very large universe of equivalent tools out there as well.

Google analytics can’t really tell you much – it’s not very actionable. The main things I like to look at are new versus return visitors, top content pages, what pages are causing bounces, etc. Again, at this stage you are still primarily driven by qualitative research and ideas, and it’s hard for analytics to drive much of your thinking.

This prototype phase might last a month or a couple months

Step 2: Traffic comes in, so data must be collected
As your product begins to mature, and you get a better sense for what you are trying to do with it, the next thing I might do is to figure out what the important pieces of data are, and confirm that it’s being measured. Nothing is worse than throwing data away that you might want to use later.

Generally, I prefer a single table or log that can be queried later that stores events. The right granularity of events is at the “business” event level, like “someone updated their profile” or “someone downloaded a video” rather than at the URL level. This ensures that you are getting a good amount of information from the logs but it’s not so overwhelming that you’re blowing up your database.

You might, for example, hold events in the rough key/value form:

user_id, event_name, value, datetime

Where it might look something like:

1000, profile.photo.update, 1, 9:30AM 3/14/2008

Make sense?

I prefer to start out via SQL so that the manipulations of the data are easy, although many large-scale systems eventually move to flat-files of some format.

Design-wise, here are some things to consider:

• What’s your “event” hierarchy and what level of granularity do you want?
• Do you want your analytics DB to be the same as your webapp DB?
• How should you join data between your webapp stats and your analytics stats?
• Where does it make sense to throw data away versus trying to store it forever?
• How do you pass data into the analytics DB? Via a JS interface called by the client (like Google Analytics) or server-side within your methods?

There’s really no wrong answers to the above – I’ve seen it done in many ways.

Step 3: Identifying your user flows
Every web product ultimately has a bunch of user flows contained within it. For example, there might be a series of flows in how users come into the site, starting with ads, SEO, or otherwise. Similarly, once they get on the site, you might be trying to optimize their usage of their site.Identifying these flows is key since you are trying to find the”critical path” that is then optimized. Figure these flows out, and make sure you’re collecting the right data to optimize.
A good place to learn about these user flows is to read about ecommerce “funnels” and how folks go about breaking those down and optimizing them.

Step 4: Trying ad hoc queries
As users are coming into the system, it can then become a good idea to start gathering data into a standard format. This means creating a small set of queries that you might try to run to learn more about the critical paths that users are taking, and where you can adjust their flow. At this point, it’s important to have the vision of the product become fairly stable so that you are starting to optimize the edges rather than reinventing the core constantly.

The kinds of ad hoc queries worth doing revolve around whatever are the tactical goals of your business. If you are trying to come up with a monetization strategy, you should try to figure out your average order size and what percentage of users that start a buying process finish it. Once you create a small list of these queries, then you can start to formalize the ideas into specific metrics that you track daily.

If any ad hoc queries return data that is similar to what you could get out of Google Analytics (for example, aggregate numbers like pageviews and uniques), it’s probably a dumb idea to try to do those in-house. Don’t do more work than you have to! Instead, the only homegrown stuff should be so specific to your business that it’s easier to do in-house than to shoehorn it into a 3rd party analytics stuff. Don’t waste your effort on numbers a off-the-shelf analytics pacakge would get you.

Assuming that your product is stable, most startups will want to tackle this within the first few weeks (but obviously not until you have data)

Step 5: Formal in-house reporting
Once the product features (and thus the user flows) are sufficiently mature to invest in this area, then it makes sense to formalize out the reports. Typically I would start out with a series of pretty plain HTML pages using tables that just print out SQL queries. You can add finishing touches like percentage %s, key ratios, etc. as you go. I generally invest zero time into cute visualizations and graphs, and prefer to read the key numbers.

How many reports should you generate? I find that it’s pretty addictive to build reports and get a clear understanding of what’s actually happening in your product. So create enough that you can make key decisions, but don’t go too far either – you’ll hit diminishing returns quickly. Generally, 2-3 reports are good enough to start, but ultimately you’ll probably track dozens of dashboards each focusing on specific aspects of your business like.

• System performance and uptime
• User acquisition via each method you use
• Aggregate metrics
• Retention
• Engagement
• Content creation?
• Ads and monetization?
• Pricing and revenue?
• etc.

Anyway, get enough data but not too much – it’s a fine balance. For timing, it probably only makes sense to do this once the product is quite stable and the key user flows are stable as well. This is likely at least a month or two out from the prototype stage.

Step 6: Too much data! Reports are too slow!
If you’re lucky, eventually your reports will be too slow. At Revenue Science, we were gathering somewhere like 1 billion pixel hits per day, and that had to be translated into reporting. Ouch. So you likely will go through a couple specific steps:

• Reports will initially query the production server – eventually this doesn’t work and slows down the site
• Reports and data are then moved off to a slave machine, where the queries still happen in real-time – but eventually this doesn’t work either because it’s too slow and there’s too much data
• Reports and data are then pre-processed every hour, and then served up – which is fine, until your queries take too long, and you have go keep moving
• Data is then replicated across a number of slave machines, where the pre-processing happens
• etc.

There are many many layers of incremental improvements you can make here – but the toughest nut to crack, in the case where your web product is HUGE is that you will be inserting more data into the system than the system can process within a reasonable time.

Then the more exotic technologies like Hadoop, HBase, Hypertable, etc start to make a difference. Most sites don’t have to deal with this so I’ll stop here!

Conclusion
Eventually, most serious analytics-driven businesses have to build their own internal analytics. It’s not pretty, but it has to be done. Hopefully the above article gives some background on the key issues you might want to look at as you scale up your product.

If you liked this blog post, please recommend it to a colleague and/or click here to get updates via email or RSS.

New mobile blog from KP
This morning, my friends at KP launched the iFundVC blog to specifically cover the iPhone and the open mobile ecosystem that’s emerging. Their first blog post details a number of investments that they’ve made in the space, in particular Pelago, iControl, ng:moco, and GOGII. Anyway, I’d encourage you to read more about this legendary firm’s perspectives on the impending transformation of the mobile landscape.

One of the important comparisons for me is between the Facebook platform versus the iPhone platform. They are two of the biggest ideas of the past few years, one to open up the monster traffic enabled by social networks, and the other to crack open the walled gardens of mobile carriers. 

Is the mindshare of hobbyist developers going to the iPhone platform?
A key issue is where the developers are going to go – most of my technical friends have spent the last year and a half hacking away building apps on the Facebook platform. However, as time as passed, the platform has grown less attractive for a number of obvious reasons – to resummarize:

• Lack of stability: Facebook may slap down your app and replace it with their own
• Lack of monetization: Beyond remnant ad networks, there isn’t much you can do with the <$0.25 CPM inventory
• Lack of investment: Many angel investors are no longer investing much into the space, as mature companies like Slide, Zynga, and others have established themselves
• Lack of market excitement: Everyone wants to find the next new thing!

I want to also note that for the hobbyists, it’s not even the money that really concerns them. They just want to tinker around with stuff and build cool products, and there’s a set of sexy features like geo-location and SMS that allow them to experiment with the interactions.

As a result, I’d argue that a class of social network “refugees” are forming who are looking to build the next new thing, and the mobile platform (iPhone and otherwise) will start looking pretty attractive for them. I wonder how many of the lessons learned on the FB platform, like social gaming, viral distribution, etc. will port over to iPhone as well.

Comparing the Facebook platform and the iPhone platform
I also asked the audience to fill in a couple of the datapoints around comparing the two platforms, and what axes they would use. Here are some of the thoughts I got back:

I’m sure there is a much more comprehensive table out there. Noah also had a fantastic table here that was very funny. Anyway, in the comments there was a fun discussion of whether or not Facebook’s huge relative audience was more important, or if having more direct monetization was more important – quite a good exchange.

Ultimately, some of the >1M adoption rates of free iPhone apps like Tap Tap Revenge show that there may be enough audience there to generate some sizeable returns. So regardless of whether or not it’s a favorable or not platform, it’s attractive enough (especially with the stronger monetization channels) for developers to get excited about it. The after-effects of the open mobile platform, across all the carriers, is likely a strong enough trend to get VCs and entrepreneurs to decide that “it’s finally here.”

Are you doing cool stuff on the iPhone platform?
If you are, drop me a note at voodoo [at] gmail, I’d love to hear more about it.

If you want to grab these in real time, you can follow me on Twitter at @andrew_chen.

As always, I can’t promise they are all work-related 

• Blog essay list is finally up to date! Added the most recent stuff… http://tinyurl.com/5m2qdq
• New site: http://AmINormalOrNot.com? My fave section is the Most Weird, with “I eat poopy” and “I have never seen star wars”
• scientific american article on how sleeping and dreaming helps you get better at tetris: http://snurl.com/3oddd 
• lots of neat aggregated stats on angel investing (via @fredwilson):http://www.angelsoft.net/industry/index.seam
• top 10 free to play publishers, representing $2B in revenue per year: http://tinyurl.com/5knx4d 
• david shen on competing via design versus features, depending on product category lifecycle: http://tinyurl.com/5jzspv 
• interview with will wright on spore “I think of the theme first, the genre last.” http://tinyurl.com/66rtk6 

Here’s the transcript:

Intro
Hi everyone. My name is Andrew Chen and I’m a blogger and entrepreneur, focused on consumer internet products here in San Francisco.
This afternoon, I gave a quick talk at O’Reilly Alphatech Ventures called “Growing Renewable Audiences” which I thought I would share.

The primary focus of this talk is about the fact that every internet startup needs to grow their audience to be successful. And for venture scale returns, you need 10s of millions of users, not 10s of thousands. So ultimately, this divides the growth efforts that you can use into two types: renewable and not renewable. I’ll get to defining what this means in the next few moments.

Techcrunch and press traffic
Let’s start with this Alexa graph. There’s really nothing special about it – they’re just another startup trying to grow their traffic. I will say, however, that this startup is one of the many that presented last year at Techcrunch 40, which leads to the question, how many of the companies that launched this week at Techcrunch 50 will look like this in a year?

My point is not to pick on Techcrunch, which I love and read on a daily basis. Techcrunch is great for getting introduced to potential partners and investors, but for a consumer internet product that’s trying to drive users to a site, it doesn’t do much. The reason is that press and blog traffic are ultimately nonrepeatable, nonsustainable audiences that doesn’t stick. You get the spike in traffic, and it melts away as quickly as it show up.

In fact, I’ll describe press and blog traffic as “fool’s gold” because of the associated emotions that it brings. It’s easy to overestimate the impact of this kind of traffic because it just feels good to have your name and company featured. It strokes your ego. You might get a bunch of inbound emails from other press and partners, and all of these things can contribute to a feeling that you’re on your way to getting tons of traffic. Problem is, you inevitably become yesterday’s old news.

So again, this is the kind of one-time traffic that I definitely discount and stop focusing on. Instead, let’s talk about what it means to build sustainable, renewable audiences.

Renewable audiences versus not
What’s the definition of renewable? I’m interested in the last part of this definition, which is to define it as “inexhaustable or replaceble by new growth.”

For a startup, this can only mean one thing: the hard-earned audiences you generate via buzz, beta testers, and other sources must beget more audiences. This means that you should focus on building repeatable, sticky traffic that will stay for the long term rather than getting the quick hits. This is the only way for startups to get big and create venture-scale returns.

So let’s talk about a bunch of methods that are renewable versus not renewable. I’ll start with the non-renewable stuff first. As I said, pr and blogs. Same thing for talking at conferences. PR, blogs, and conferences are great to attract investors, partners, potential employees, but terrible for trying to scale to 10s of millions of users.

Same for community building events like meetups, contacting influencer communities, and so on. This can help you build out intuition for your product, but it won’t help you grow your userbase to 10s of millions.

And finally, here in San Francisco we have the “cult of feature worship.” Every product must have better features than the next, and it’s easy to respond to issues of traction with thinking. But I’ll argue that features might increase your engagement, but have a tough time driving more users.

Compare this to the renewable strategies, like viral marketing, SEO, widgets, and ads, which can scale into 10s of millions of users but are primarily centered around tough, non-user centric work. These are things that if you get right, you can optimize your way into a big, sustainable audience.

So I’ll stop here and ask you: What strategies is YOUR company using? Are they more from the left hand column? Or the right hand column?

Taking a systematic view to growing audiences
Finally, let’s talk about the approach for how you execute these growth efforts and build up sustainable audiences. First, you need a mental model for how users enter your site, and the process in which they bring in other users. This is your growth funnel, which you should then measure in extreme detail, and then use A/B testing tools to optimize. If you approach this process scientifically, then you’ll end up generating a vast array of hypotheses which you then identify, measure, and optimize.

I’ll give an example of this in advertising, and why ads can be a renewable resource.

An example: Growing audiences using ad arbitrage
Let’s say that you have a product that has a great LTV backend, like a free-to-play MMO that uses virtual goods for monetization. Then ultimately the entire focus of your business should be around figuring out how to buy ads profitably. get them to your site, upsell the experience so that a % of users buy. This then enables you to focus on buying more advertising, which lets you reach more users. If successful, you’ll be able to grow your audience up to its maximum size, until you run out of ads you can arbitrage effectively.

I updated the Community Next sldies from last time with an audio track, so I thought I would publish these slides again. Pardon the intrusion if you feel like you’ve already grokked all the details!

Intro
Hi everyone. My name is Andrew Chen and I’m a blogger and entrepreneur, focused on consumer internet products here in San Francisco.

I recently gave a talk called People Not Pages at the Community Next, which I’ll share today.

This year, the online advertising industry has reached roughly 20 billion dollars, that’s a big number. And a big chunk of those dollars are here because of the fact that advertisers can target consumers in ways that are simply not possible in traditional media.

Talking about ad targeting
So today we’ll talk about the business of ad targeting, and in particular 3 things:

First off, how can publishers generate more revenue from ad targeting? How does this actually happen?

Secondly, how do advertisers evaluate the effectiveness of their targeted campaigns? By understanding how they think about the question “does it work?” publishers can serve them better as customers.

And finally, we’ll discuss a topic that’s near and dear to my heart, which is: Why do social networks suck so much at monetization?

Note that this talk will mostly focus on the publisher-centric view, since the startups here in San Francisco tend to be creators of ad inventory rather than buyers of it.

About me
So quickly, before we begin, a quick bio about me. Again, my name is Andrew Chen and I’ve spent the bulk of my career in online advertising. I was most recently an Entrepreneur-in-residence at Mohr Davidow Ventures, a time I greatly enjoyed, evaluating deals in ad infrastructure as well as companies that are ad-supported. Before that I spent 4 years running product at Revenue Science, a behavioral targeting ad company in Seattle, and before that in the automotive vertical at Cobalt Group.

About Revenue Science
Most my experience will be drawn from my time at Revenue Science, so I’ll talk briefly about the company. Basically, as a behavioral targeting ad provider, they work with some of the largest brands online – Reuters, NY Times, ESPN, AOL, and dozens of others – to help them increase their revenues via targeting. While I was there I worked on their brand publisher products as well as getting their ad network off the ground.

Generating more revenue as a publisher
On to our first topic: How does targeting generate more revenues for publishers?

To understand this, I think you have to go all the way back to the newspaper days. This is an article from the early 1900s, costing one cent, documenting when the Titanic struck an iceberg. You have to think about how advertising was bought and sold at that time to understand why things work the way they do online today.

Now imagine this: You’re a newspaper company, which means you actually have to print physical copies of your text. That means that if you’re going to sell advertising, you must also print your ads on that physical copy, next to the editorial that’s relevant. That means that as an ad sales professional, your job is to take an audience that an agency specifies, let’s take “small investors” as an example, and translate that into a particular section in your paper. And so you sell them the Finance section. This means the agency asks for people, but you sell them pages.

The modern day online publishers are not much different. For a publisher like NY Times, you’ll have a bunch of sections on your site. You have the homepage, which makes a good CPM because it gets the most traffic and is the most prestigious placement on the site. Then you have high-value sections like Technology, and Finance. Then you have low value sections like Opinion and World News, which are harder to sell because what advertiser wants to be next to a story about the War in Iraq? So all of the low value stuff gets classified as “remnant ad inventory” and is lumped together.

So, this means that you have a bunch of different sections, and you can sell each for different prices. Again, the agency asks for a specific kind of person, let’s say IT decisionmakers, and you give them back the Tech section.

But wait a minute here, the IT decisionmaker reads the tech section, but also browses around the site. Is it right to value them highly on one pageview, and then drop them on the next? This is really a vestige of the old newspaper mindset.

If the agency is fundamentally targeting an audience, and you can deliver this audience regardless of where they are on the NY Times. or potentially even across the internet, you can deliver them more value. And lucky for the publisher, they are then converting their low-CPM remnant inventory into high-value targeted inventory.

This process of finding the qualified audiences in the remnant inventory is really what a publisher strives for.

Targeting techniques
So all the different targeting techniques that are out there, be it Geo, SIC, Demographic, Content, Daypart, or Behavioral, is all about finding these valuable audiences.

Once you have your targeting toolset in place, then the value of your ad inventory is increased. You find the IT decisionmakers in the opinion section, and convert low-value ads into high-value ones.

Evaluating targeted ad campaigns
Great, now let’s move to the second topic. How do advertisers evaluate whether or not targeting campaigns are working?

To understand this topic, you have to understand that there are really two major groups of advertisers out there: Brand advertisers, and direct response. Let’s talk about each in succession.

Brand advertising metrics
Brand advertisers are the guys you see all over the place, like Coke, Budweiser, Honda, United Airlines, and others. For many of them, their biggest goal is to drive awareness of their brand messages regardless of strict ROI. Obviously the ads are meant to eventually sell product, but it’s often not accountable from a strict ROI basis.

This brand spend means big dollars. Take a Budweiser NASCAR vehicle, for example. These cars can cost up to $20MM to sponsor. This year, Facebook is rumored to gross $300MM in revenue, which only translates to 15 NASCAR sponsorships.

So what kind of metrics are they looking for?

Well, here’s a case study that I worked on many years back. American Airlines and WSJ worked together on a campaign which was evaluated using Dynamic Logic, an ad effectiveness company. They went and surveyed users who had seen the campaign and who hadn’t, and asked them questions around message awareness and purchase intent. And it led to some positive results, which I’ll discuss.

First, there was a metric called Audience Composition. The campaign was targeted at frequent business travelers, so the survey qualified who fit the bill. The targeted campaign had a higher % of business travelers than the untargeted one, in fact by more than 2X.  Same for the definition of heavy travelers, people who took more than 5 trips a year, where it was 145% better.

The other set of metrics were brand and persuasion metrics, like Aided Brand Awareness and Message association. For example, those questions might revolve around things like, “Of the following brands, which do you associate with higher comfort” and would name a series of airline brands. The ones in the campaign were correctly associating the brand message.

Direct response advertising metrics
Now let’s go to direct response advertising. I used a picture of a cheesy infomercial because that’s the kind of advertiser that really cares about their results. Every dollar they spend must generate more dollars. They care a lot about ROI, as a result.

The example case study here, which I did not work on, is from Yahoo. In their example, they created an audience segment of mobile users who searched for ringtones, specific mobile providers, and browsed relevant pages. For the advertiser targeting this audience, the campaign generated higher CTRs and conversion rates, which are good ROI metrics. The better the increase in conversions, the more you as the advertiser can spend on this audience to get them to purchase your products.

Social network monetization (and why it’s hard)
OK, and the final topic for today, a discussion of why social networks suck so much at generating revenue. This is a topic I’ve written a lot about on my blog.

First off, this slide shows what you get when you search for “myspace addict” in Google images. The fact is, people find MySpace insanely addicting, which is great for them on a pageviews and engagement perspective. Problem is, this same engagement is what makes MySpace users pretty uninterested in what you’re trying to sell them. They are simply not in the mood to buy stuff, and when social network audiences are in their “flow state” clicking around, the CTRs can be quite low. This is exactly what you see, with CTRs often 0.1% and below, whereas search CTRs are often in the 10s of percentage points.

Social network data = sucks for targeting
The next issue is the type of data social networks are amassing. I included a real profile from a friend of mine, Andrea, which states “I have one interest: Dane Cook. I’m kidding I have more: Movies, music, soccer” etc. This tells you that Andrea’s interested in these things, but not that she’s in the transactional window to buy something. This is completely different than the kind of data you get from users when they are searching for dane cook movies on Google, or searching for dane cook on Amazon.

This issue is really the distinction of Interest versus Intent, and it’s a really hard area to break through. There’s a lot of social advertising companies working on this problem, and good luck to them, because it’s very hard.

Don’t assume that all the data you’re collecting is useful, but it’s likely that it’s not, unless the user is very close to making a transaction.

Wrap up
OK great, so we’ve covered the main topics. We talked about how publishers make money, and the process of turning remnant into premium. We also talked about brand versus direct response metrics in evaluating targeting campaigns. And finally, we talked about why social networks are so bad at monetization.

If anyone has questions, please feel free to e-mail me or check out my blog. Thanks!

Comparing the iPhone platform versus Facebook platform
I’ve recently been pondering the key similarities and differences between the way the Facebook platform ended up getting executed versus the iPhone platform. It’s an interesting trend now that other mobile players are thinking about opening a platform and providing an app store as well.

How would you guys add to the following table?

What else should be in this table?

Do users really love your product?

If they did, how would you be able to tell?

I would argue that the single most telling metric for a great product is how many of them become dedicated, repeat users. This angle of thinking naturally leads to a number of metrics around user retention, which we’ll examine in this blog post.

User retention is especially important for social web products. Failure to consider the backend retention of a userbase can lead to catastrophic results – in particular, without the proper mechanics in place, it’s easy to hit the “shark fin” user curve, as well as the death spiral caused by reverse Metcalfe’s Law. In both cases, once the core audience of a site starts to erode, then the erosion can cause a negative feedback loop that causes the entire audience to fall away.

This raises a series of questions:

• What are the right metrics to track for user retention?
• (And as a corollary, what are the wrong ones?)
• What is a “good” retention number? What are bad retention numbers?
• How do you optimize and improve retention rates?

Let’s tackle these below.

Retention versus Engagement
First off, there’s an important distinction between engagement versus retention, which some folks often track in one bucket. I generally define retention is simply the act of getting users BACK to revisit, regardless of their actual activity on the site. Contrast this with engagement, which measures how much time they spend with the product, how many features they interact with, etc.

An implication of this is that the right metric to follow is visits rather than something like pageviews or time-on-site.

Here are a couple examples of the separation of engagement versus retention:

• Google is a high retention, low engagement site
• MySpace is a high retention, high engagement site
• News sites are often medium/high retention, low engagement sites (like checking a headline)
• etc.

Note the important point that engagement doesn’t necessarily correlate with monetization. Because many retail sites and reference properties are transactional in nature, oftentimes this implies that the closer you are to the money, the lower the engagement is.

Keep this in mind for people who espouse “addictiveness” and “engagement” as virtues for social media sites.

Retention versus Acquisition
Secondly, there’s the important issue of how to disambiguate newly acquired users from retained users. The problem with a traffic graph that’s going up-and-to-the-right is that it’s not clear what’s really happening – is the site bringing in lots of new users? Or is there a bunch of dedicated users that are extended their engagement? You need to figure out which of 4 scenarios are actually happening, which I’ve blogged previously about:

1. Pageviews are coming ONLY from new users
2. Pageviews are coming ONLY from one generation of users (like early adopters)
3. Pageviews are coming ONLY from retained users
4. Pageviews are coming from new users and retained users

The proper way to disambiguate retention from acquisition is to precisely track the following stats:

• How many new users are joining the site?
• Of these new users, what are the different funnels they are joining from? (be it SEO, direct navigation, etc.)

Then you separate out these users completely from the aggregate numbers, and the remaining folks you have left are ones who are coming back to the site. You can then further segment this group by cohort, which we’ll discuss below.

Building your first retention table: User cohorts vs Revisit rates
Using the points from above, you can now build a retention table that compares how many users are coming back. This table starts with three columns:

• Time period the user joined
• Number of users that joined that period
• Revisit percentage rate

The reason why you separate it out into cohorts is that it gives the ability to compare performance of the site over time. As new product features are added, ideally the revisit rates would also continue to rise.

Let’s put this together in a table, imagining that we’re at Week 5:

A couple points on the above table:

• Looking back as Week 5, you can see that Week 1 is now the “oldest” cohort, and those users have had many weeks to revisit the site
• The overall userbase is growing 10% per week, starting with an initial userbase of 1000
• The revisit rate is naturally <100% since whatever initial cohort you start out with, it can only decrease but not increase
• Note that the retention rate of the site seems to be around 30%, although you’d want to let the Week 1 cohort run for a while and see if it eventually stabilizes
• Week 5 is currently at 0% since in this example the week just started and no users have revisited yet
• The actual number of visits on any given day is weird to calculate using this table, since the view is not based on aggregate numbers

The key metric is really the number that the revisit rate converges to. You can use this number in your traffic models to understand whether you should be focused on acquiring new users, or if you can simply focus on extending the engagement levels of your site.

What’s your revisit rate? (Using Google Analytics to approximate it)
Google Analytics gives you an overall number for free, with some caveats. You can access this feature on the lefthand nav through “Visitors”, then “New vs. Returning.” Basically this is an OK approximation of the revisit rate, as long as you:

• Maximize the window in which you are doing the analysis (ideally starting the analytics window when the site was first made public), otherwise the numbers will skew high since you’ll be counting too many dedicated users
• Ideally, the site would isn’t adding exponentially more users every day, since it would skew lower because newer users are less likely to have returned

Essentially there’s some skew that comes into play since Google Analytics doesn’t let you segment your users based on when they first joined the site.

Willing to share?
For readers who are willing to share the numbers on their site, please comment below and if I get enough responses I’ll do a followup blog post on the subject.

Like this blog?
If you did, please recommend it to a colleague and/or click here to get updates via email or RSS.