Andrew Chen Archives

Subscribe · Featured · Recent · The Cold Start Problem 📘
Dear readers, I have moved to Substack and I will be writing here from now on:
In the meantime, I will leave up for posterity. Enjoy!

How do you do concrete interviews for non-technical people?

Why do people interview technical and nontechnical people differently? Nerds vs jocks

Interviewing engineers and objective tests for competence
Recently, I’ve been meeting a number of engineers to find more people I might want to work with. I think as a whole, most tech companies do a pretty good job interviewing engineers because at the very least, there are objectively correct answers to programming questions. In particular, you can give engineering questions which result in code that either runs or doesn’t, and having an interview candidate code with you for an hour is pretty enlightening.

(Note that ultimately, there’s still lots of gray area, and some solutions are better than others, but there’s at least a minimum bar for objectivity. You can never get rid of human judgement, of course)

Because there’s a strong signal for competence, as a result, you can create a series of “can you tie your shoelaces?” type questions which quickly suss out their level of expertise, and you can do this all in one phone call (or at the very least at the end of a couple hours together). In fact, if you search for “programming interview questions” you can get a sense for how concrete these interviews get – I think this is great.

Nontechnical interviews and weak signals for competence
Now let’s compare this to nontechnical interviews, which, in my expertise at least, generate weak signals for competence: Almost every interview process I’ve ever been involved with, whether I’m the interviewer or the interviewee, seems to lack the level of rigor that most engineers go through. Why is that? I imagine that much of it has to do with the fact that in nontechnical positions, it’s harder to decide objectively what’s “good” or “bad” – people often disagree on strategy, design, and it’s hard to figure out if you’re actually competent or not.

As a result, many of the nontechnical interviews I’ve seen tend degenerate into descriptions of previous work, or soft skills, or very subjective conversations around “how would you improve X or Y?” That’s not to say that these discussions aren’t useful, but for me personally, I’ve seen far too many people with 10 years of experience in some area that turn out to not be able to tie their shoelaces. The question is, how do you find this out sooner rather than later?

In fact, it may be that this entire discussion isn’t really about interviewing for technical people versus nontechnical, bur rather thorough interviewing versus sucky processes. Even then, I’d mostly argue that there’s a real issue that you can use objective tests in the engineering world to create strong signals of competence, whereas it’s much harder for marketing and product roles.

Crafting concrete interview questions for nontechnical roles
So what would a series of concrete tests look like for nontechnical roles? I would argue that you can rigorously test a couple key areas such as:

  1. Can you create the deliverables that are part of the day-to-day role?
  2. Are you familiar with previous relevant work in your area (whether you follow it or not)?
  3. Can you demonstrate that you can do the thing you’re being hired to do?

Let’s drive into each one of these areas, as a thought experiment of what it’d look like to do an interview structure that’s as concrete as what most engineers have to go through:

Part 1: Can you create the deliverables that are part of the day-to-day role?
This is probably the closest that you can get to a coding question, at least for nontechnical people. The point is, most nontechnical jobs still do provide deliverables to other people in the company – for some, they will be spreadsheets, or documented product roadmaps, or launch schedules, or powerpoints, or whatever. The question is, can you have them sit down and craft a basic version of whatever deliverables they’ll be expected to create on the job?

Here’s an example: Let’s say that you were going to hire a product manager who needs to have a strong background in user acquisition via search engine marketing. Ideally, you should be able to sit them in front of a blank spreadsheet and they should be able to model out the user acquisition process from start to end. This means they’ll know how to think about the problem like a funnel, show the different steps, be able to roughly approximate what the numbers might be, and then calculate the cost per acquisition.

Or, if you have you interviewing for sales, they should be able to sketch out the basics of an RFP response, or build out a sales pipeline document, or make a list of sales collateral they might need, or whatever. If it’s someone from the marcomm world, then they should be able to sit down with you and craft a budget for making a splash at a tradeshow, or creating a schedule for a product launch, or whatever.

The point of all of this is that it’s a “tie your shoelaces” exercise that complements the soft skills discussion and meandering conversation about previous work.

Part 2: Are you familiar with previous relevant work in your area (whether you follow it or not)?
The next set of questions can be asked around how engaged they are in previous work in their area, regardless of whether or not they follow it. This area I’m often torn about, since there are often talented people who don’t know anything about historical precedence – but I do think that it demonstrates competence in the main. In concrete terms, I think that you can test for a couple specific things:

  • Are they familiar with industry jargon in their field?
  • Do they understand the theoretical underpinnings for what they’re doing?
  • Have they read relevant books and blogs, attended conferences, or otherwise engaged in the discussion?

So for example, a product manager who focuses on go-to-market strategy should ideally be familiar with books like Crossing the Chasm or be aware of previous successes/failures in the tech industry. If the product manager is involved in the development process, you’d want them to be familiar with Scrum or Agile development, and ideas like the man-month. If they involved in product design, they would ideally know terms like visual language or affordances or Fitt’s Law.

As with the caveat in the previous section, I would ask these questions primarily to suss out expertise level and while it would contribute to a final hire/no-hire decision, it wouldn’t be the overriding factor. Ultimately some people are amazing decision makers on products without having formal training, but as entrepreneurship has a long history of failures, you’d ideally find people who were familiar with other situations that led to success or failure.

Part 3: Can you demonstrate that you can do the thing you’re being hired to do?
Similar to a programming interview to test programming skills, ideally you’d have the applicant tested in a way that most resembles their actual day to day job. That way you are testing them for their actual skills, rather than their self-reported skills. The trick to this, I think, is to break down the actual job description into specific areas that define the success or failure for them in the role.

For example, let’s take hiring a technical recruiter, whose day-to-day role you might break into:

  • Prospecting (finding new candidates)
  • Making contact and selling
  • Evaluating skillsets
  • Scheduling and interview coordination
  • etc.

Ideally, the applicant would be tested using the real tools that they would use. If they are prospecting and using Linkedin, you’d give them a job description and ask them to pull up the site. Then you’d have them go through and try to find good candidates for you. Similarly, you’d ask them to pick a particular candidate and draft a high-quality, personalized request for them to come in. And so on.

When my girlfriend interviewed for IDEO, the global product design consultancy, they had her present her resume to a group of people in slide format and then take questions from a group. This is really smart, because of course a lot of her job is to take ideas, synthesize them down, and present it in groups. So I think that having her do that is a great test for competence in this area.

What’s next here?
I think the next step in this blog discussion would be to actually post some job titles and give rough formats for interviewing for that type. If I have time over the next couple weeks, I’ll write something up.

Has your company created a rigorous process for selecting non-technical hires? If so, I’d enjoy hearing more – please write a comment.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

PS. Get new updates/analysis on tech and startups

I write a high-quality, weekly newsletter covering what's happening in Silicon Valley, focused on startups, marketing, and mobile.

Views expressed in “content” (including posts, podcasts, videos) linked on this website or posted in social media and other platforms (collectively, “content distribution outlets”) are my own and are not the views of AH Capital Management, L.L.C. (“a16z”) or its respective affiliates. AH Capital Management is an investment adviser registered with the Securities and Exchange Commission. Registration as an investment adviser does not imply any special skill or training. The posts are not directed to any investors or potential investors, and do not constitute an offer to sell -- or a solicitation of an offer to buy -- any securities, and may not be used or relied upon in evaluating the merits of any investment.

The content should not be construed as or relied upon in any manner as investment, legal, tax, or other advice. You should consult your own advisers as to legal, business, tax, and other related matters concerning any investment. Any projections, estimates, forecasts, targets, prospects and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Any charts provided here are for informational purposes only, and should not be relied upon when making any investment decision. Certain information contained in here has been obtained from third-party sources. While taken from sources believed to be reliable, I have not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation. The content speaks only as of the date indicated.

Under no circumstances should any posts or other information provided on this website -- or on associated content distribution outlets -- be construed as an offer soliciting the purchase or sale of any security or interest in any pooled investment vehicle sponsored, discussed, or mentioned by a16z personnel. Nor should it be construed as an offer to provide investment advisory services; an offer to invest in an a16z-managed pooled investment vehicle will be made separately and only by means of the confidential offering documents of the specific pooled investment vehicles -- which should be read in their entirety, and only to those who, among other requirements, meet certain qualifications under federal securities laws. Such investors, defined as accredited investors and qualified purchasers, are generally deemed capable of evaluating the merits and risks of prospective investments and financial matters. There can be no assurances that a16z’s investment objectives will be achieved or investment strategies will be successful. Any investment in a vehicle managed by a16z involves a high degree of risk including the risk that the entire amount invested is lost. Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by a16z is available at Excluded from this list are investments for which the issuer has not provided permission for a16z to disclose publicly as well as unannounced investments in publicly traded digital assets. Past results of Andreessen Horowitz’s investments, pooled investment vehicles, or investment strategies are not necessarily indicative of future results. Please see for additional important information.