Andrew Chen Archives

[Adapted from an answer I wrote on Quora, and thought I’d share it on my blog too.]

Adding a lot more features won’t save your product
Everyone’s worked on a product it’s failing despite a ton of work behind it. It’s not for lack of great ideas, or a lack of bright minds working long and hard on the product. In the startup world, often this comes because after a new product is launched, there’s a Trough of Sorrow where features are often added to try to spark traction. After a few months of this, and a few shifts in direction, it’s easy to get a Frankenstein product that tries to do too much.

At this point, adding new features won’t help– what’s broken is at the core of your product, not out on the edges. Adding more to edges won’t do anything, because most of your users aren’t even getting there.

Eric Ries has a wonderful term for what to do here, which is to consider a “zoom in pivot.” He talks about it in his book Lean Startup, as a kind of pivot you can do if your product isn’t gaining traction.

The idea of the zoom in pivot is:

A single feature in a product becomes the whole product, highlighting the value of “focus” and “minimum viable product,” delivered quickly and efficiently.

The question is, how do you pick the feature you’re going to zoom into? And how do you validate that it can work as a standalone product? And how do you execute the pivot itself and what metrics can you look at?

Picking the new product
The actual process of picking the new product is the same as picking any new product for a startup. Ultimately it still has to go after a huge market, it has to be differentiated against competitors, and have a distribution model. You have to be passionate about it. Etc, etc. All the standard strategy issues apply, and I’ll leave this as an exercise to the reader.

In terms of tactics though, the big thing from a metrics standpoint is to try and figure out what’s actually getting enough usage to actually execute the “zoom in” pivot. After all, if you zoom into a smaller featureset that isn’t being used currently, that’s obviously much risker than noticing that out of 10 features, 1 or 2 are getting all the usage, so then you dump everything else.

Based on developing a product strategy, and looking at current usage metrics, you can develop a hypothesis for what a smaller product might look like. You can also create some goals you want to hit as far as the metrics are concerned- obviously the usage of the zoomed in feature should be much higher, but by how much? And the usage of the secondary features should become zero or minimal- are you OK with that? The next step is to test it.

Iteration and testing
It should be easy to test a “zoom in” pivot- just default the navigation and the description of the product to focus on what you’re zooming into. You can even test a few ideas simultaneously if you want to.

Here are a few high-impact places to test:

• Changing all the landing page where new/unregistered users arrive to reflect the new positioning
• Taking users directly into the functionality after they sign in or sign up, so that you are defaulting to that usage
• Using modal lightboxes or other highly prominent UI to channel users into the zoomed in featureset
• At the end of the typical workflow of the user, to take them to the feature again

The above suggestions focus on making the zoomed in feature more prominent, but you can also make the other features more secondary. You can do the following:

• Burying other features into submenus like “Extras” or “Goodies”
• Removing other features from global navigation UI
• Rewriting headlines to de-emphasize unneeded features, or removing text about them from landing pages, bulleted lists, etc.

The combination of all of the above – either by making the main feature more prominent, or the burying the secondary features – should help the goal. You can A/B test these, primarily focusing on new users, to see what the effect looks like.

From a metrics standpoint, I think as a baseline you’d want the zoomed in feature to increase significantly in usage, and for the secondary features to go to zero or nearly so. You also want to make sure some of the aggregate stats around frequency of use, time on site, content shared, etc. to be stable depending on what you care about.

Choosing a feature
After this iteration process, picking the zoomed in feature should be easy. You may have to go through an A/B testing process to smooth the transition from the old featureset to the minimalist one, but over some period of time you should be able to make the metrics move in the direction you want.

If it turns out the metrics are stubborn and some important metrics go down, then that’s much more problematic. It might turn out that the zoomed in feature you picked is somehow not right enough. Or maybe the userbase you’ve amassed isn’t right for the pivot. Or maybe you need to develop the featureset a bit more, in the direction you’ve pivoted, to get to the right product.

For all of these, the Plan B might be to either accept the new featureset and deal with the reduced numbers, hoping to fix them later. Or alternatively, the Plan B might be to pick a new featureset or continue iterating on the zoomed in featureset, until it works. That’s all gray area.

This is an oldie but a goodie I had to share again- here are some screenshots of early web products, some close to their inception, some a couple years later.

If you have some other screenshots of early products, be sure to tweet me at @andrewchen and I’ll try to put them up.

[My friend Lars is a product marketer at KISSmetrics and loves helping SaaS businesses understand how their business is growing. He writes regularly for the KISSmetrics blog and his personal marketing blog. He wrote the following post about SaaS products and the metrics you use to evaluate their success level. Lots of great information in there. You can follow Lars at @LarsLofgren -Andrew]

How healthy is your SaaS business?

We’re bombarded with KPIs and an endless series of metrics to tell us how we’re doing.

But instead of using data to measure our progress, it’s much more likely that we get lost and start focusing on metrics that are easy to track but don’t mean anything.

For a SaaS business, there are a few core metrics that need your undivided attention. And the priority of these metrics shift as you grow. If you’ve only had paying customers for 2 months, it doesn’t make much sense to track lifetime value. But later on, lifetime value is essential.

In this post, I’m going to break down the essential metrics for each stage of a SaaS business.

What this framework will give you:

• By focusing on a few key metrics, you’ll also be focusing on the core problems you need to solve to get your business to the next level.

• Data doesn’t do you any good unless you act on it. Each of these metrics clearly tells you how you’re doing. Right away, you’ll know where you need to spend your time.

• Each stage has two metrics that balance each other. This keeps you from over-optimizing one metric and unintentionally harming the long-term health of your business.

Let’s jump in.

Before Product/Market Fit
You’ve just made the decision to start your business and you’ve got plans for world domination.

But before you can start building your empire, you need to make sure you have the right product for the right market.

For most new products, there’s usually a disconnect at the beginning and customers don’t quite want what you have. Either you need to go after a different target market or you need to change your product to fit their needs. When you get this match, we call it product/market fit.

You’re probably in this stage if:

• You’ve just started.

• You don’t know who your ideal customer is.

• People are testing your product for the first time.

This is the first major hurdle you’ll need to overcome. But how do we measure our progress when we don’t have any data? You don’t even have any paying customers at this point and if you do, it’s not many. At this point, running a bunch of A/B tests won’t help you test your business model.

Instead, you’ll rely heavily on qualitative feedback and one critical survey question.

Primary Goals:

• Validate core business assumptions by talking to people in your target market. If people ask you for your product before you even try to sell them, you’re going in the right direction.

• Survey users and have at least 40% say that they‘d be very disappointed if they had to stop using your product.

Metric #1: Qualitative Feedback
Yes… this isn’t technically a metric. But it’s too early for data anyway so you’ll need to make the most of what you can get: feedback.

Right now, you really only have one goal: build the right product for the right market. And the fastest way to do this is to start talking to your customers.

If you have any users at this point, jump on Skype and get a deep understanding of their main problems. Ask them to show you how they currently solve the problem you’re going after. Then show them what you’ve been working on to see if they get excited about it. Usually, you’ll want to follow this format for the interview:

1. Basic demographic questions to get a better sense for who you’re talking to.

2. Deep questions about the current problem.

3. Present your solution for feedback (don’t sell it, just get feedback).

You’ll want to do 10-20 of these customer interviews.

If you don’t have any users at this point, go and talk to people that you think would want to use your product. This is a great way to start testing different target markets efficiently. It’s a lot easier to schedule 10 more Skype meetings than it is to rebuild or rebrand your product.

When you want to start scaling feedback (especially as you move into the later stages of your business), use Qualaroo surveys, SurveyMonkey, feedback forms, and usability tests like UserTesting.com. But when you’re just starting, talk to people in your target market one-on-one. The insights will always be much better.

At KISSmetrics, we still do customer interviews each and every time we make a major change to our product. Adding a new feature? Go talk to customers. Revamping an old feature? Let’s talk to our customers that use it the most. Starting a new project like our Google Analytics app? Find a group of Google Analytics users to talk to. We do it every single time.

Metric #2: Measuring Product/Market Fit
There’s just one little problem with all this customer feedback though.

It’s super difficult to measure objectively. Are people REALLY interested in our product or are we only focusing on the positive feedback while downplaying the negative feedback?

Luckily, there is a survey question that will help you quantify whether or not you’ve reached product/market fit. Full credit goes to Sean Ellis for this question. Here it is:

How would you feel if you could no longer use [product]?

1. Very disappointed

2. Somewhat disappointed

3. Not disappointed (it isn’t really that useful)

4. N/A – I no longer use [product]

Send this to people that have used your product at least twice, experienced your core product offering, and used it in the last two weeks. The goal is to get at least 40% of your users to say “very disappointed.”

If you don’t meet the 40% benchmark, you may need to reposition your product or pivot entirely. If you do hit it, time to move on to the next stage.

More Resources
To dive into more detail on what you’ll need to make it through this stage, read through these posts:

• Marc Andreessen’s post on product/market fit (this is an archived page, the original was removed from his blog).

• This breakdown of customer interviews will walk you through all the details.

• Experiment with these methods to scale your customer feedback.

• A great deck from LaunchBit on how to find customers to interview.

Beginning to Scale
So you’ve found product/market fit.

You’ve got revenue coming in and a growing customer base. Now it’s time to build a business.

Up until this point, you didn’t really need to track much. Outside of basic user signups and revenue, there wasn’t anything to track. Now that you got the right product for the right market, there are two metrics that will keep you headed in the right direction.

You’re probably in this stage if:

• You’ve found at least one way to acquire customers consistently.

• Many of your customers stay subscribed and want to keep paying you.

• Your monthly revenue is starting to grow.

Primary Goals:

• Consistently grow MRR while controlling churn.

• Get monthly churn to 1-2%. If it’s above 5%, ignore everything else until you lower it.

Metric #1: Monthly Recurring Revenue (MRR)
For a SaaS business, monthly recurring revenue is a much more valuable metric to track than traditional revenue. It’s the total revenue you received during the month that came from recurring subscriptions.

The health of a SaaS business heavily depends on recurring revenue. It can take months to regain the cost of acquiring a customer and the real profits come from increasing that subscription revenue. One-time windfalls just aren’t that valuable to us. By tracking monthly recurring revenue, we can see exactly how our business is doing month-to-month.

Unfortunately, tracking MRR can get tricky. There’s several use-cases that your tracking systems will need to be able to handle:

• Having annual plans on top of your regular monthly plans complicates things a bit. The annual revenue actually needs to get divided between each month of the subscription, not just the month when the customer is billed.

• Upgrades and downgrades get tedious to track. If a customer moves from a $10/month plan to a $50/month plan, you’ll need to add an extra $40/month to your MRR.

• You’ll need to remove revenue when it churns with a cancellation.

Speaking of churn…

Metric #2: Churn
Growing MRR is one side of the coin at this stage. The other side is churn. If you can’t keep customers subscribed, it won’t be long before your MRR won’t budge and your business will stall.

The thing is, churn can be a devious metric. At the beginning, a monthly churn rate of 10% doesn’t seem so bad. If you have 100 customers, 10 of them left. Not that big a deal right? It’s pretty easy to get 10 more customers. But what happens when you have 10,000 customers? Now 1,000 of them left in a single month. Even the best marketing machines have a hard time keeping up with something like that.

Your churn rate starts out innocent and easy to handle. But it can quickly get out of control if you’re not keeping a close eye on it. In order to build a strong foundation that will help your company grow over the long-term, you absolutely, without a doubt, NEED to get control of your churn rate.

So what’s a good churn?

It always varies by industry. But in general, it’s critical that you get your monthly churn under 5% and your goal should be 1-2%. Later on, you can experiment with upsells and cross-sells to get negative churn.

Expansion
Sooner or later, you’re going to hit a wall.

The main channel you’ve been using to acquire customers will start to slow down and you’ll hit diminishing returns. If you want to keep growing each month, you’ll need to find new sources of growth.

You might start testing affiliate programs, new ad networks, PR, business development, referral programs, new types of content marketing, conferences, event marketing, or whatever type of marketing happens to be hot at the moment. You’ve got LOTS of options to choose from. Some of them will be a great fit for your market, others will fail completely.

You’re probably in this stage if:

• Growth is beginning to slow for the first time.

• Continuing to improve your main channel is getting a lot harder.

• You’ve successfully controlled your churn.

So as you start to experiment with new channels of growth, you need to focus heavily on two metrics. These metrics will keep your experiments in check and make sure you scale profitable channels.

Primary Goals:

• Keep your cost per acquisition to one third of your lifetime value.

• Get each customer to profitability within 12 months.

Metric #1: Lifetime Value (LTV)
How much revenue do you earn in total from a customer before they leave your business? For a SaaS business, it’s absolutely critical to track lifetime value. When you factor in acquisition, support, and product costs, it can take a SaaS businesses 6-12 months to turn a profit on a customer.

To make sure customers stay long enough to keep your business healthy, we use lifetime value (some people abbreviate it as CLV or LCV).

By now, you’ll have had customers long enough so that you can actually figure out your LTV. Use the formula here to get started. When you have more resources, you might also want to include second-order revenue in your LTV calculation.

Metric #2: Cost Per Acquisition
As we begin to experiment with new channels to keep growing, cost per acquisition keeps us in check. It’s the total cost it takes to acquire a customer from a particular source.

For the average CPA of your business, you can total up your entire marketing and sales expenses over a month then average that out over the total customers you acquired. But we need to take it a step further and segment CPA by acquisition channel. This tells us whether or not customers from new channels are worth the effort.

When you’re experimenting with new channels, it’ll usually be pretty obvious if the math won’t work out. Bad channels tend to be BAD channels. So keep experimenting until you find the ones that work.

Not only will CPA help you evaluate new channels for growth, it’ll help you figure out how far to push your main channels. How much can you actually spend to acquire a customer on AdWords or Facebook? How many writers can you hire to put together content? By keeping an eye on these metrics, you’ll know how far is too far.

A popular rule of thumb is to keep CPA to one third of your LTV. And a customer should become profitable within 12 months.

More Resources
Once you get past product/market fit, use these posts to help you work through all the details:

• Use this list of metric definitions from David Skok for your formulas.

• Also read through Skok’s SaaS Metrics 2.0 post which is probably the best post on SaaS metrics I’ve come across.

• Here’s a great SaaS KPI dashboard from Christoph Janz.

• Why you shouldn’t worry about applying a discount rate to your LTV as a startup (the post focuses on mobile gaming but it’s applicable to any startup).

• Some of the downsides of focusing on LTV too heavily.

• David Cancel’s framework for building a data-driven startup (with a bit more emphasis on funnels and cohorts).

A Quick Overview
For each stage of your SaaS business, track these metrics:

1. Before Product/Market Fit: Customer feedback and the product/market fit question

2. Beginning to Scale: Monthly recurring revenue and churn

3. Expansion: Lifetime value and cost per acquisition

Keep in mind that each stage is not completely exclusive. Let’s say that I’ve found product/market fit and I’m starting to scale. If I’m using AdWords to acquire my customers, I’ll definitely want to keep an eye on my cost per acquisition. But at this point, I’m still trying to get a handle on my churn for the first time. I don’t REALLY know how long these customers are going to stick around. So I’ll check my CPA to make sure it’s somewhat reasonable (if the total revenue from a 12-month subscriber doesn’t cover it, you have a problem). Otherwise, I’ll spend most of my time improving MRR and churn.

What about funnels? What about engagement metrics, ARPU, active users, number of visits to signup, and everything else? By all means, track the other metrics you need. But the above metrics are the bare minimum. Move mountains to track them before worrying about the rest. There’s little reason to track a random engagement metric if you don’t know what your MRR or churn is.

What have I missed? I’d love to know how you track your own SaaS business.