[Andrew: Paid marketing remains an integral part of many products’ acquisition channels, and one of the key metrics is Cost of Customer Acquisition, which is a nuanced calculation with lots of gotchas. My good friend Brian Balfour (ex-vp growth at Hubspot) put together this incredible essays with details on how to think about it.]

Brian Balfour (ex-VP Growth at Hubspot):

In everything from growth projections to company valuations, it’s common to use CAC and CPA interchangeably — but it’s wrong, and it can cost you. In this post, I break down growth’s most important jargon to demystify the true cost of acquisition, and dig into the most common mistakes leading growth teams off track.

Note: This is a big topic that’s best addressed with live examples and interactive frameworks. To that end, I’ve included a number of examples of real companies, plus interactive spreadsheets that you can access throughout the guide. To adapt any spreadsheet for your own calculations, click here, then go to File > Make a copy to create your own version.

Customer acquisition is not CPA – Three examples

To start off, let’s address a common myth. Customer acquisition cost (CAC) and cost per acquisition (CPA) are commonly conflated, and yet in reality they’re completely different metrics. Understanding the difference is the start to understanding CAC in depth.

CAC specifically measures the cost to acquire a customer. Conversely, CPA (Cost Per Acquisition) measures the cost to acquire something that is not a customer — for example, a registration, activated user, trial, or a lead. The two are related because CPA is usually used to measure the cost of things that are leading indicators to CAC.

Four examples of how CAC and CPA are different but related:

1. Dropbox

Since Dropbox is a freemium product, CAC would be the cost to acquire a paying user on either their pro or their team plan. CPA would be used for things such as the Cost Per Registration (of a free user), Cost Per Activated (free) User, and other important, but still non-paying, actions that signal that someone has moved from being a visitor to a user of the product.

2. HubSpot

Since HubSpot is a B2B SaaS product, CAC would equal the cost to acquire a new customer on one of their Basic, Pro, or Enterprise plans. CPA would be used for leading indicators to CAC, such as Cost Per Lead, Cost Per Sales Qualified Lead, Cost Per Trial or other points in the marketing and sales funnel.

3. Facebook

B2C companies supported by ad models are a little different. In Facebook’s case the paying customer are advertisers so CAC is the cost to acquire a new advertiser.  However if you just look at users, users/customers are the same. CPA is most likely used for things like Cost Per Registration, Cost Per Activated User, etc.

The key point is the first thing you need to do to understand CAC is very simple:

Define who/what a customer is in your model.

Make that definition clear and simple and get the language between CAC and CPA consistent otherwise communication will be very confusing.

The basic calculation of CAC and why it’s wrong

If you google ‘how to calculate cost of customer acquisition” you will get the basic formula below:

CAC = Total Marketing + Sales Expenses / # of New Customers Acquired

On the surface this is correct, but it is missing a lot of details and definitions around each variable in the equation to get it right. Even the best basic calculation can be very misleading. For example, using this formula you might look at the below spreadsheet and assume the following:

To adapt this spreadsheet for your own calculations, click here, then go to File > Make a copy to create your own version.

But what if I told you the following things:

1. It takes on average for most customers 60 days from lead to become a customer.
2. Not all customers are new customers, but some of them are returning.
3. This is a freemium product and there are costs to supporting users while they are free before they become a paying user (customer).

These three tidbits of additional information should change how we look at the basic formula above. Instead of taking the equation at face value, we need to evaluate a few questions first.

How to really calculate CAC

There are three key questions we need to ask to define a more accurate calculation of CAC for a business. All of them dig into a level deeper around the variables in the equation.

Key Question #1:  How long between your marketing/sales touch points and when someone becomes a customer?

The first issue with the basic calculation is that it doesn’t take into account the time period between when you spend the marketing/sales money and when you actually acquire a customer. Here are two examples:

Example 1:  Freemium Product

Let’s look at Dropbox as an example. When you sign up for Dropbox you start using their free tier. You use Dropbox free for some time period until you hit your storage space limit, which you then might upgrade. For a lot of users that time period is months (and in some cases over a year). The story is the same with other freemium products such as Evernote, Buffer, etc.

Example 2:  SaaS Company w/ Inside Sales

Most SaaS companies with inside sales models someone might become a lead this month (due to our marketing efforts this month) but will take 60+ days before they become an actual customer because they need to go through the sales process.

If you don’t take these time periods into account, you could be overestimating or underestimating CAC and as a result making some terrible operating decisions.

Here is an example of how you could overestimate.In the below example, CAC is being calculated by taking the month’s marketing costs and dividing it by new customers in the same month.

To adapt this spreadsheet for your own calculations, click here, then go to File > Make a copy to create your own version.

In March we try some new channels which causes a large increase in marketing costs.  Under the simple calculation our CAC is $148.  If our target CAC is $125, we would likely make the decision that March was unsuccessful and we would turn off those new channels.

But, let’s say it actually takes 2 months for someone to become a customer.  Below is the same data but changing the calculation to account for this 2 month period.

To adapt this spreadsheet for your own calculations, click here, then go to File > Make a copy to create your own version.

This change tells a completely different story. In March we have a CAC of $84, and in April a CAC of $111. Instead of turning off the new channels we tried in March, we would probably make the decision to scale them.

The key question about time between marketing/sales expenses and acquiring a customer does not matter if you fall in one of two scenarios:

1. The time between marketing touch point and someone becoming a customer is very short. This is true for a lot of B2C companies that have very short decision funnels for users: Snapchat, Instagram, and others.

2. Your marketing/sales expenses are so consistent that it normalizes itself out over time. But even in this case it is best to be more accurate.

You need to figure out how the timing of the expenses correlate with the timing someone actually becomes a customer. Marketing expenses could correlate differently than sales expenses.

The simplest way to account for this is figure out your average marketing/sales cycle. In other words what is the average amount of time from first marketing touch point, to acquiring the customer?

Here’s an example. Let’s say we are a SaaS company where the average time is 60 days from lead to customer and we believe the sales expenses are spread evenly over that two month time period.

The CAC equation would be as follows:

CAC = (Marketing Expenses (n-60) + 1/2 Sales (n-30) + ½ Sales (n)) / New Customers (n)

n= Current Month

Here is an example model with that calculation built in:

To adapt this spreadsheet for your own calculations, click here, then go to File > Make a copy to create your own version.

Key Question #2:  What expenses do you include in the Marketing + Sales?

The second question you need to answer to get an accurate CAC calculation is what expenses do you include in the numerator (marketing/sales)? Before we look at some examples of the answer to this question differs, here are the three most common mistakes I see:

Mistake #1:  Not Including Salaries

You need to include the salaries of all people working on marketing and sales. We see CAC numbers a lot where they aren’t included. This includes not just the individual contributors who are 100% dedicated to marketing/sales, but also those (often times managers) who spend part of their time on marketing/sales. A CAC number with salaries included is often referred to as “Fully Loaded CAC.” There are some scenarios where it is useful to separate Fully Loaded from Non-Loaded CAC which we’ll explain later.

Mistake #2:  Not Including Overhead

Similar to the mistake of not including salaries, you need to include the overhead (rent, equipment, etc) allocated to those employees working on marketing sales.

Mistake #3:  Not Including Money Spent On Tools

The marketing and sales tool space has exploded. Most teams are using 10+ tools to operate their marketing and sales machine. These tools can add up in cost and need to be included in the expenses  of your CAC calculation.

If you include those main things, the answer to this question starts to get a little more complicated and range from company to company. A couple of examples:

Spotify and the Case of Freemium – Does that Include Product/Engineering/Support?

Spotify is a freemium business. They have millions of users using the free version of their product which helps them acquire new users through sharing music and other viral channels.

In most companies, product, engineering, and support are not included in CAC (typically part of R&D). But if the free product is your primary method of customer acquisition, shouldn’t the expenses that support that free product be included in the expenses portion of your CAC calculation? There are different opinions to this question, but we fall on the side of yes.

If you had engineers, PMs, and other roles on the marketing or sales team for marketing/ops you would include those salaries and expenses in the CAC calculation. The engineers, PMs, or other roles might not technically be on the “marketing” or “sales” teams, but they are still expenses that are required to support new customer acquisition.

HubSpot and the Case of SaaS –  Include Customer Success Costs?

Most SaaS companies like HubSpot have sizable Customer Success teams. The definition and roles of these teams can vary widely. Some Customer Success teams are purely devoted to churn prevention. Some are devoted to helping onboard a new customer. Some are devoted to winning previous lost customers back.

All this begs the question, should customer success costs be included in CAC? Some of these responsibilities touch acquiring new customers and therefore make the case that they should be allocated to CAC.

Dollar Shave Club and the Case of Subscription Ecommerce – Support, Shipping for Free Trial?

Dollar Shave Club is a subscription ecommerce business. They famously have a one month trial for $1. That one month trial has a number of expenses outside of marketing, including shipment costs for the initial package, support during the trial, and other costs.  Should those costs be included in CAC? The answer to that depends how you define a new customer.

In Dollar Shave Club’s case, we would make an argument that someone on a $1 trial is not a customer yet. A new customer is someone that extends beyond the trial.  Therefore all of those costs associated with supporting the free trial should be included in CAC. Those costs are partially offset by the $1 trial payment, but not fully offset.

A customer is a customer, right? Not necessarily. When it comes to calculating CAC, we need to distinguish between new and returning customers. In most organizations there are marketing and sales efforts focused on new customers, and there are marketing and sales efforts focused on retaining or getting customers back.

The mistake is only including marketing and sales expenses in the numerator for new customers, but including all customers (including returning customers) in the denominator. This will make your CAC look artificially low. You can solve this in one of two ways:

1. Include all marketing/sales expenses (including those focused on retention) and all customers.

2. Separate expenses for new customers from reactivating old customers and separate out new from reactivated customers in the denominator.

Recap

From the examples we’ve covered today, I hope it’s more clear that calculating true CAC involves a lot more than a simple, one-size-fits-all equation.

Instead, an honest assessment of customer acquisition cost looks at the length of your sales cycle, how many customers are truly new customers (as opposed to returning customers), and the total costs and resources required to support marketing efforts that lead to new customer acquisition.

Brian Balfour is CEO of Reforge, previously VP Growth @ HubSpot, EIR @ Trinity Ventures, Co-Founder of Boundless, Co-Founder of Viximo, Co-Founder reCatalyze & PopSignal. 

[Andrew: Excited about today’s guest post! I was recently interviewed by the folks at Reforge, a new company started by my friends Brian Balfour and Susan Su focused on advanced professional education. They asked a great question – how do you interview for growth folks? I gave some my 2 cents based on my experience helping startups and growth folks. They pulled together a great essay below!]

Growth Interview Questions

Guest Essay by Susan Su, Reforge

Together, Shawn Clowes (Atlassian), Elena Verna (SurveyMonkey), Nick Soman (Gusto), Andrew Chen and Brian Balfour (Reforge, previously at Hubspot) have interviewed or screened over 1,000 individual candidates for growth roles – both for their current and previous companies, plus startups where they’ve invested/advised.

Growth is an emerging field, and there’s hardly a playbook on how to ace your growth interview (whichever side of the table you’re on), and yet hiring and team could be the most important “growth hack” of all. I recently asked a handful of folks from the Reforge Collective about the questions they ask when interviewing candidates for competitive positions in growth.

Here are some of the questions we’ll cover:

1. The Golden Gate Bridge
2. Growth Hacking a City
3. Are you an early adopter?
4. Live brainstorm an experiment backlog
5. What Are Your Setbacks?
6. Look for opportunity, not just risk
7. But… why?
8. The Bullshit Test
9. The Trajectory of Growth
10. Probe on Metrics
11. And on Culture
12. 6-Month Roadmap
13. Growth Resources
14. Low Hanging Fruit

If you’re looking for your next job in growth, or you’re trying to fill one at your company, consider this a peek into how a few other growth marketing experts are structuring their growth interviews.

This 1 Weird Interview Question
Growth is as much about showing creative problem solving as much as quick metrics-based iteration. If a candidate is out there googling answers to “trick” interview questions, then they’re missing out on an opportunity to showcase their own unique way of combining creativity with numbers.

“Weird” interview questions let you out to roam beyond the constraints of the role in front of you. The best weird questions are designed to highlight original problem-solving, and if that doesn’t excite you, then a position in growth probably won’t be fun or fruitful.

1. The Golden Gate Bridge

When she was still a candidate interviewing at SurveyMonkey, Elena Verna fielded an unusual interview question that she continues to use in her own growth interviews to this day:

Let’s say you were to come in tomorrow, and you got a new project to work on the Golden Gate Bridge and its toll collecting. But, you’re told that they’ve recently lost all of the tracking of both traffic flow and revenue collection, and your boss has just told you that you need to come up with a revenue estimate in the next 15 minutes. You don’t know anything about how much has been collected historically. All you have is a blueprint of the bridge.

How would you estimate how much revenue or the bridge generates on a weekly basis?

Elena explains that the question is meant to demonstrate your problem solving skills (not to get to the “right” answer necessarily).

For me it’s a sign that you’re willing to tackle an open-ended question in a creative way — without all of the data and answers. How am I going to approximate with only broad strokes of knowledge? What are the key variables (Bridge length? Car length? Car weight? Traffic volume?), and how far can I go in 15 minutes?

Elena insists there’s no right answer. Instead, it’s a window into your process of thinking with uncertain variables — and also how excited you get when you’re tasked with reasoning through an unsolved problem.

2. Growth Hacking a City

Ultimately, hiring for your growth role is both high risk and high opportunity. The teams with the most at stake (and offering the best opportunities) will need to explore every last corner of a person’s thinking and style, and that sometimes means taking a lateral approach.

Nick Soman poses an unusual question designed to get at the way candidates think about growth beyond the templates and playbooks that circulate in the growth community:

How would you growth hack a city?

It’s not an immediately technical or product-based experience, and yet it’s an interesting question that might actually become more and more relevant. How would you attract residents to it? How would you attract the other people and elements that that ecosystem requires? What mechanisms would you employ to grow your city? It’s very revealing to see how people approach growth when they have no templates, when they start from zero.”

3. Are you an early adopter?

Have you ever thought about your own relationship with Snapchat? Have you broken down your own psychology of engagement with Facebook? How about for lesser-known, non-consumer products that are in your life?

We are what we eat, and the products and apps we consume (and how we interact with them) can say a lot about who and how we are when it comes to creating and growing our own products and apps.

Shaun Clowes wants to know what you’re using at work:

If you just got a new computer at work, what apps would you immediately set up?

I’m looking for their take on a piece of software that they care about, something that gets them excited, and then how they explain it to me.

What are the most recent apps you been playing with on your phone?

That gives me insight into how in touch you are with the industry, how much you’re seeking out things that are different or somewhat common, and whether you’re an early adopter of things.

4. Live brainstorm an experiment backlog

It’s the default to come prepared to a growth interview. You’ve looked at their core growth loops, you’ve analyzed their funnels, and you have an (externally informed) idea of where the business is going. But what if you were put on the spot to dig even deeper into how the business can grow?

Nick Soman wants to see candidates live-brainstorm an experiment backlog:

How many ideas can you come up with in 3 minutes?  Maybe it’s a handful, let’s say 5. Then, I really want to push for a 6th or a 7th.

I want to see the candidate beyond their comfort zone and extend beyond their pre-planned ideas and analysis, especially in real-time. I resist the urge to ask follow-up questions — even when I’m really curious — because I want to see you go for breadth, and then I want you to be able to follow that up with an objective assessment of those ideas.

5. What Are Your Setbacks?

If none of your experiments are failures, then you’re probably not testing the right things. Running growth means swimming in setbacks — and it helps if you’ve had some life experience with that.

Andrew Chen says the level of your setbacks say more about you than their specific details:

One of the questions that I like a lot that doesn’t have to do with growth but tells you a lot about a person is to ask about someone’s biggest setback, either personally or professionally. You can get a sense of if that’s a real setback and how they reacted to it.

For example, someone who’s very junior is going to come up with a small setback. They’re going to say, ‘Oh well at work I did a project and it sucked.’ That’s very different than something like, ‘I was at a company and I convinced the board to do something and it was a long decision and the company failed. And everyone got fired.’ Or, ‘I moved from country A to country B, left my family and friends and started over again.’

Understanding setbacks helps you understand what a person’s priorities are, and how much resilience they have to bounce back.

What Most Growth Interviews Are Missing

6. Look for opportunity, not just risk

A lot of people try to understand a candidate’s weaknesses, but Elena Verna wants to know your superpower. Early to mid-stage growth is as much about doubling down on unique advantages as it is about fixing leaks (ie, identifying and addressing weaknesses). Later, the truly stand-out orgs are the ones defined by their unfair advantages, particularly in growth and product.

Building a growth team is a microcosm of the trajectory you want to see that team take, according to Elena:

Very few growth managers will look at an opportunity in a person and say, “Ok, these are your strengths and I’m actually going to tailor a roll around you to make sure that I’m playing to your strengths.

It’s good to know what people aren’t good at — where they’ll be a liability. But I want to dig into to what they’re excellent at as well. That’s what you really need to focus on, and to make sure that that strength aligns with the position at hand or that it’s possible to mold the position around it.

Too often, we identify a problem or a hole in the business and start looking for the person that will fit it. The person you find could be effective very early on, but evaluating too tightly against specific role can be very short-sighted. Yes, they might be able to sort out that immediate issue for you but in the same stroke you may end up hiring the wrong person long term.

The real opportunity is finding the person who will be happy (and make your business happy) as the definition of growth itself expands, and the immediate problem becomes obsolete. Where do you want them to be in a year? Look for the opportunities, not just the “urgent” holes.”

7. But… why?

Most of us aren’t professional interviewers. We know our area, or we know growth marketing as a broader domain, and we stick to what we know. But, in building a growth team, you’re called to ask people about things that may not be your cup of tea.

As a result, many of us don’t dig into the details of a candidate’s experience, which is a bad idea for both the hiring organization and the candidate — the former misses out on opportunity and risk assessment (is this person a good fit?), and the latter may not get to tell their punchline.

Shaun Clowes asks a deeper layer of question where most interviews have called it a day.

Most interviewers will ask you a question about how in the past have you done X thing. You give them a surface-level answer like ‘We did Y and then achieved Z.’

A few ways to drill in more deeply would be to follow up with questions like:

• When you say “we,” how much was you and how much was everybody else?
• Were you really pivotally involved in this or was this really something that you just got carried along with?

Sometimes it feels like the answer has been rehearsed. It’s the correct answer, but when you drill into it, it’s clear that either the initiative isn’t all they said it was or it wasn’t as deep as they said it was or their involvement wasn’t as deep as they said it was. The best way to get through this is with a one-word question: ‘Why?’ ‘Why did you do that?’

You actually can reply with “Why did you do that?” to every subsequent answer, and it’s almost endlessly educational. This addresses the need for depth that growth roles need, but that many interviews often lack.”

8. The Bullshit Test

How do you know someone really knows growth and doesn’t just have a great handle on acronyms?

Growth is a critical role but not one that hiring teams can succinctly test for. That is, you can’t check out someone’s GitHub for growth or decouple what public evidence you find of their work from other situational factors, like a great team, a solid company, or a unique ecosystem opportunity whose bandwagon they jumped onto.

Andrew Chen runs a “bullshit test” to make sure that candidates aren’t merely fluent in blog posts and jargon.

I ask people to get on the whiteboard and draw out the whole thing. For example, I may ask someone “How does YouTube grow?

There, I want to watch you draw out the entire flow for how a user comes into YouTube and how you think it might all work — just from what you’ve observed on the outside. Then I’ll ask you where you might make improvements. I want you to do all of this in real-time with me in the room.

This exercise shows a level of detail and thinking that indicates that you’ve mastered what you’re doing, versus that you’ve read all the blogs. I want to get the sense that your capacity goes beyond knowing the concepts, and that you have a depth of process understanding that you can bring to anything you do next.”

What You Should Ask Your Interviewer (but Probably Aren’t)

“Do you have any questions for me?” is a common wrap-up to an interview session. It’s also its own covert test of your listening skills and the depth of your analytical abilities.

But aside from generalities about company culture, project overviews, and basic metrics, the top candidates for growth roles get under the surface of their opportunity with specific questions for their interviewers.

9. The Trajectory of Growth  

The definition of growth or growth team can differ significantly from one business to another. Some growth teams are focused only on driving acquisition into the business while others are making fundamental calls on product strategy and development.

Elena wants to see candidates who are taking a long, non-static view of growth.

Ask, ‘What does growth mean for this company, and what will it mean?’ You need to know whether they are responsible for driving metrics across the rest of the funnel or not, and how they may or may not evolve with the rest of the business.

Ask, ‘How does the growth team actually catch up with the structure of the business as it evolves?’ Many applicants simply want to understand where they’re going to be in a couple of months. This is very short-sighted. It’s not just about growth today, in this place and time; it’s about trajectory.”

10. Probe on Metrics

For growth roles you want to know are you coming in to fix the broken system, or are you coming in to make a good system great?

Understanding the answer to that fundamental question comes down to understanding both the metrics of the business and the culture of the team.

When it comes to metrics, most people simply don’t go far enough into the specifics. Brian Balfour says that’s where great growth candidates stand out.

What does retention look like? What does LTV look like? What are the biggest dropoffs?

If a growth candidate doesn’t ask me those basic question in an interview, I’m shocked.

But then you need to keep going. Keep asking questions about the metrics until your interviewer stops you and says, “It’s too detailed and we can’t give that out in an interview”.

You should get as much information as you possibly can. Not only will you know what type of situation you’re walking into, you’ll also show that you know how to think about growth for that company.”

11. And on Culture

The velocity of growth is determined by one part strategy, one part implementation. Great strategy and promising metrics can still be blocked by cultural issues within the organization; successful growth is technical, but it’s also fundamentally human.

Brian wants to see candidates who seek to understand the relationship between the growth team and other teams: core product, marketing, sales, executive.

Ask, ‘If I wanted to make X type of change in this part of the product, what would the process look like to make that happen?

Follow it with, ‘Would I or my team have our own resources and autonomy to be able to make that change?’

Changes can require negotiation, politics and navigating a number of other ‘people’ steps. You need to pose that same type of situational question to your interviewer that they’re probably asking you: ‘If I wanted to do this we thought if it was a good idea to do this, how would we get this done?’

That gives you a much better idea of how the team works, rather than simply asking them, ‘What’s the process around here?’

You’ll start to realize when they are describing how to get something done that there are certain points where they’ll show discomfort. Those are the areas where there’s friction within the company.

No company is perfect. But, it’s much better to know what the flaws are going into it, rather than being surprised after the fact. That way you can be better prepared and more effective from day one.”

12. 6-Month Roadmap

Being effective in flux starts with having a sense of what’s expected of you in the first six months in a new role. When you’re running weekly experiments or solving previously untackled problems, Andrew Chen says you should ask:

If I were to join, what would I be tasked with achieving in the first 6 months?’ You have to have a good sense for the 6 month roadmap — what you would actually do in terms of experiments and goals — so that you can come in with that from day 1.

13. Growth Resources

There are many different flavors of growth, and resourcing the growth function is a key variable from team to team. Growth has been interesting because there are different flavors of it.

Andrew wants to see candidates ask:

Are there are going to be dedicated engineers? Are there going to be dedicated designers? Or is this a situation where we need someone to kind of think about growth but they’re not part of the product?

14. Low Hanging Fruit, aka How Many Times Has the Homepage Been Optimized?

The best candidates want to understand the potential reaches of their own impact. How wide and deep are the outcomes that are under your purview? Andrew wants to know how much low-hanging fruit already been picked.

Have people been working on growth or really smart people thinking about it?

As a candidate, I would want to ask how many times has the homepage been optimized in the last 6 months, same with landing pages, same with everything. That gives you a sense of the kind of impact that you’ll be able to create.”

Susan Su leads marketing at Reforge providing training and connections for growth professionals, and is a venture partner at 500 Startups.  Special thanks to the Reforge Collective members who contributed to this post: Elena Verna, Nick Soman, Andrew Chen, Shaun Clowes, and Brian Balfour.