Andrew Chen Archives

Subscribe · Featured · Recent · The Cold Start Problem 📘
Dear readers, I have moved to Substack and I will be writing here from now on:
In the meantime, I will leave up for posterity. Enjoy!

Are your SEO efforts working, or failing?

(For a quick summary of Search engine optimization, check out the Wikipedia entry)

SEO is a black art – with an emphasis on Art not Science
If you’ve ever wanted to increase website traffic, you’ve likely stumbled on a dozen or more guides on SEO. In fact, if you google "SEO tips" and look specifically for PDFs, you end up with this results page. Click through a couple and check them out.

In general, you’ll find that most of the SEO articles seem to consist of a long list of rules for what’s "good SEO" like:

  • Put the right keywords into your URL
  • Change your META keyword list to include important phrases
  • H1 is so much better than H2
  • Use sitemaps, because that helps your indexing
  • … blah blah

OK, I’m sure that most of these work, at least to some degree. And at the very least, large groups of consultants and agencies benefit from this confusion, because it ends up looking like a tax code – if you don’t know the ins and outs of hundreds of possible rules, then it’s hard to think about tackling this in-house, or even evaluating the performance.

Like viral marketing, the vast majority of the knowledge out there consists of little lists of "rules of thumb" ideas to implement. And also like viral marketing, there’s a lot of tribal knowledge and hucksters who make a living writing about cute ideas, without putting it all together into a coherent theory.

If SEO were a science…
Let’s think about how the medical sciences has evolved over time:

  • 15,000BC to 1700s AD:
    • Hmm, if you’re sick and you eat god-knows-what, you seem to get better!
    • Also, you might decide to make up some rules for WHY it is that something happened
    • This could be, of course confusing correlation with causation
  • 1746 AD:
    • British guy does the first ever A/B test to figure out that citrus cures scurvy
    • Of course, you still don’t know why – but it works
  • 1853 AD:

And of course, it was only recently that we started to understand HOW and WHY certain drugs work, through a deeper understanding of the human body and the chemistry involved.

So back to SEO, if the accumulated knowledge essentially consists as a bunch of "best practices," then what you have is a very nonscientific approach to the problem. It’s basically at the stage of eating various things, seeing if they kill you, and collecting a list of plants that may or may not help you. And as a result, you’d have a list of "Eat this, and not this"-type rules, rather than a unified way of approaching the problem.

What are the right SEO metrics?
First off, if anyone has found some interesting data or articles in this direction – please send them to me – I’d love to read more about the topic.

In the meantime, I’ll talk about what I’ve found so far: After searching around, I definitely found some interesting articles for how to incorporate these ideas into a broader framework – here are two:

Both are worth a quick read.

The idea is to separate out different parts of SEO. You have 3 components:

  • Methods (do X not Y, do A not B)
  • Performance dashboard (our # of pages crawled is X, let’s get it to Y)
  • A/B testing

So generically, you basically have the actual experiment you are running – like whether or not a change you make to the site helps – and then an experimental framework that helps validate what is happening.

As a result, you end up breaking down a performance dashboard that has variables like:

  • Number of pages crawled
  • Number of pages indexed (3 Main Engines)
  • Number of days X % of the pages survive in Google from first full crawl to deindex (What I call ‘Burn Rate’)
  • Indexing methods used
  • Uniques
  • Page views
  • Clicks
  • Revenue

(quoted from SEOIdiot)

Or, metrics like:

  • Brand-to-nonbrand ratio
  • Unique pages
  • Page yield
  • Keyword yield
  • Visitors per keyword
  • Index-to-crawl ratio
  • Engine yield

(quoted from Stephan Spencer)

But having all of this stuff together is pretty confusing, isn’t it? Especially when you can’t figure out what question you are really trying to answer?

Putting these metrics into a dashboard
Tracking a large collection of metrics is not interesting in itself. What you are essentially looking for is a series of steps – I’ll frame this a community site that generates SEO pages, since I’m sure that’s what a lot of people are working on:

  1. User comes to the site
  2. User writes an article
  3. Article is then published to a unique URL
  4. URL is then crawled
  5. URL is then indexed
  6. Keywords are then searched
  7. URL then shows up in the search result page
  8. URL is then clicked on
  9. Then a user ends up back on the site

What you basically want to do is to step through and track variables against each step of this process – the more granularity you can get, the better.

In fact, let me restate the really interesting question you’re trying to answer from these stats:

For each page created by a user, how many additional
users its likely to attract over its lifetime?

Integrating this into a
holistic dashboard, you’re then able to see how the changes you make
impact the variables throughout the site.

However, that’s not what most people do – they might track only steps #1,#9, through aggregate pageview numbers. Or if they are going to track steps #1,2,3,9, since those are the only things happening directly on their site.

But of course, in the case where you can’t measure directly, you can measure indirectly via using comScore or Compete data, or doing external searches yourself, or whatever. Or, you start using measures like in-bound links, or some other metrics, which lets you at least roughly estimate how a change you make alters the numbers for what Google will give you.

If someone has developed or used a dashboard like this, I’d love to talk about it ;-)

This still leaves A/B testing, which I’ll leave to a thought experiment for the reader, or a future blog.

UPDATE: My good friend and old college roommate, Eric Peters, gives his take here. Eric is a metrics guru and was previously at MSN working on monetization, so his opinion is absolutely worth reading.

PS. Get new updates/analysis on tech and startups

I write a high-quality, weekly newsletter covering what's happening in Silicon Valley, focused on startups, marketing, and mobile.

Views expressed in “content” (including posts, podcasts, videos) linked on this website or posted in social media and other platforms (collectively, “content distribution outlets”) are my own and are not the views of AH Capital Management, L.L.C. (“a16z”) or its respective affiliates. AH Capital Management is an investment adviser registered with the Securities and Exchange Commission. Registration as an investment adviser does not imply any special skill or training. The posts are not directed to any investors or potential investors, and do not constitute an offer to sell -- or a solicitation of an offer to buy -- any securities, and may not be used or relied upon in evaluating the merits of any investment.

The content should not be construed as or relied upon in any manner as investment, legal, tax, or other advice. You should consult your own advisers as to legal, business, tax, and other related matters concerning any investment. Any projections, estimates, forecasts, targets, prospects and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Any charts provided here are for informational purposes only, and should not be relied upon when making any investment decision. Certain information contained in here has been obtained from third-party sources. While taken from sources believed to be reliable, I have not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation. The content speaks only as of the date indicated.

Under no circumstances should any posts or other information provided on this website -- or on associated content distribution outlets -- be construed as an offer soliciting the purchase or sale of any security or interest in any pooled investment vehicle sponsored, discussed, or mentioned by a16z personnel. Nor should it be construed as an offer to provide investment advisory services; an offer to invest in an a16z-managed pooled investment vehicle will be made separately and only by means of the confidential offering documents of the specific pooled investment vehicles -- which should be read in their entirety, and only to those who, among other requirements, meet certain qualifications under federal securities laws. Such investors, defined as accredited investors and qualified purchasers, are generally deemed capable of evaluating the merits and risks of prospective investments and financial matters. There can be no assurances that a16z’s investment objectives will be achieved or investment strategies will be successful. Any investment in a vehicle managed by a16z involves a high degree of risk including the risk that the entire amount invested is lost. Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by a16z is available at Excluded from this list are investments for which the issuer has not provided permission for a16z to disclose publicly as well as unannounced investments in publicly traded digital assets. Past results of Andreessen Horowitz’s investments, pooled investment vehicles, or investment strategies are not necessarily indicative of future results. Please see for additional important information.