Andrew Chen Archives

Subscribe · Featured · Recent · The Cold Start Problem 📘
Dear readers, I have moved to Substack and I will be writing here from now on:
In the meantime, I will leave up for posterity. Enjoy!

Designing and Testing an Ad Product: 5 lessons learned form imeem’s audio ads (Guest Post)

Another guest blog, today from Sachin Rekhi, who is currently an entrepreneur-in-residence at Trinity Ventures. You can find more of his writing at, and follow his Twitter at @sachinrekhi. Most recently, he ran product efforts in Imeem’s monetization efforts, and before that started the company through the YCombinator program (acquired by Imeem). I asked him to write about some of his work monetizing Imeem’s ad inventory, and some of the issues he worked through – his post below is about those projects. One last important note, Sachin is engaged to marry my sister Ada next year! Congrats to the happy couple ;-) –Andrew

Designing and Testing an Ad Product: 5 Lessons Learned from imeem’s Audio Ads
By Sachin Rekhi

In its search to find the most effective way to monetize user’s time spent listening to music, imeem has become one of the early innovators in the nascent online audio advertising space.

From the process of designing, testing, and iterating on imeem’s unique audio ad product, I wanted to highlight 5 key lessons learned that are applicable not only in developing imeem’s ad offering, but in general to designing any innovative ad product.

Lesson 1: Align the ad product with your site’s user experience
Lesson 2: The easy way is often not the best
Lesson 3: Pick the right metrics to optimize
Lesson 4: Make sure to look at qualitative feedback
Lesson 5: Iterate on the sell in addition to the ad product

Align the ad product with your site’s user experience

imeem had classically employed a variety of advertising strategies to monetize users, including display ad inventory that was filled by our direct sales team through high impact brand campaigns as well as dozens of ad networks we used to fill our glut of remnant inventory.

Yet we knew with our audio consumption experience, we were creating a new kind of available ad inventory which could be much more effective at reaching our users than display ads since audio-based advertising better aligned with the activity users were most engaged with on the site. With terrestrial radio ads still generating $21B in revenue, there was clearly an opportunity to shift some of those dollars online and provide a better experience for both users and advertisers.

The easy way is often not the best
Online audio ads are not a new concept. They have been used by a variety of major online streaming outlets, including AOL Radio, CBS Radio, Live 365, and Yahoo LaunchCast. However, the initial incarnation of audio ads took the easy way out. They typically ran 30 second audio ad spots which they obtained from ad agencies that re-purposed their terrestrial radio creative for online audio ads. This made it very easy for agencies to get their feet wet with online audio advertising with no additional creative costs. While this may work for traditional online streaming services, the new generation of music streamers like imeem, Last.FM, and Pandora would not be willing to run such long audio ads out of fear of losing their user base.

So what was needed was an audio ad unit custom tailored for personalized streaming services. And that’s what we ended up creating at imeem. We came up with an 8 second audio ad spot that would advertise a national brand and show a standard IAB medium rectangle (300×250) banner on top of the player during the audio ad playback. The user could click-through the medium rectangle to the advertiser’s landing page like classic banner ads. We started with a very low frequency of a maximum of 2 ads per user per hour.

However, this was far from easy, as it required imeem to develop in-house production capabilities for the 8 second audio creative, as agencies never had existing creative and were rarely willing to develop another set of creative themselves. While this was an undertaking, it is often necessary to bear the cost of innovation to deliver the right ad product to your audience.

Pick the right metrics to optimize
In order to understand the effectiveness of any ad unit, it’s important to systematically test it. The first step in designing a successful experiment was determining what were the metrics that we were testing. We knew that we were trying to satisfy two customer segments with this ad product: advertisers and users. For advertisers, there were a variety of ad-related performance metrics that we could measure. However, we decided to start by measuring the advertiser metrics that ad agencies had classically been most interested in. We wanted to determine whether we could make advertisers happy through the performance of these classic metrics, since trying to educate ad agencies on the importance of new metrics is an uphill battle that would significantly decrease your ability to sell the unit. Thus the initial advertiser metrics we tracked were click-through rate of the tethered medium rectangle banner as well as aided and un-aided brand recall as measured through quantitative surveys administered by our research partner Dynamic Logic.

For users, what we wanted to understand was whether introducing audio ads onto our site would decrease the amount they used the site. While we tracked page views, visits, session length, etc, we focused on number of songs played per user during the life of the experiment as the most important proxy for site usage.

Make sure to look at qualitative feedback
In addition to measuring quantitative metrics, it’s equally important to collect qualitative feedback from real users. The iModerate online focus groups we conducted ended up being very enlightening and allowed us to derive interesting insights of consumer motivations and behaviors that looking at the quantitative data alone wouldn’t provide.

For example, though initially we were significantly worried that the introduction of audio ads would cause users to flock to our ad-free competitors, we learned through interviews that many of our young users had developed a strong affinity with imeem, understood the need for imeem to monetize, and were eager to suggest ad verticals they would be most interested in hearing to improve the product.

Iterate on the sell in addition to the ad product
An area that’s as important to iterate on as the ad product itself is how you sell or position this offering in the marketplace. Selling innovative ad products is actually the greatest challenge in the process. Anytime you introduce a new ad unit, significant education is required for brand marketers and agencies to help them to understand the importance, effectiveness, and promise of this new medium.

Our sales planning team iterated many times on the pitch to advertisers for the audio ad product as well as how we reported on ad unit performance at the end of each campaign. This was regularly refined based on feedback we elicited from our advertising partners.

While many have claimed the death of online advertising in light of the recession, its important to remind ourselves that ad dollars are still being spent online. Now is an opportunity to innovate on the ad products that we offer advertisers to show greater value, brand awareness, and performance. We must keep in mind that ad agencies are eager to find better ways to spend ad dollars, as they are equally interested in showing results to their brand clients to hold on to their ad budgets. We should partner with our advertisers and users to find the most efficient way to leverage online advertising to monetize our sites.

PS. Get new updates/analysis on tech and startups

I write a high-quality, weekly newsletter covering what's happening in Silicon Valley, focused on startups, marketing, and mobile.

Views expressed in “content” (including posts, podcasts, videos) linked on this website or posted in social media and other platforms (collectively, “content distribution outlets”) are my own and are not the views of AH Capital Management, L.L.C. (“a16z”) or its respective affiliates. AH Capital Management is an investment adviser registered with the Securities and Exchange Commission. Registration as an investment adviser does not imply any special skill or training. The posts are not directed to any investors or potential investors, and do not constitute an offer to sell -- or a solicitation of an offer to buy -- any securities, and may not be used or relied upon in evaluating the merits of any investment.

The content should not be construed as or relied upon in any manner as investment, legal, tax, or other advice. You should consult your own advisers as to legal, business, tax, and other related matters concerning any investment. Any projections, estimates, forecasts, targets, prospects and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Any charts provided here are for informational purposes only, and should not be relied upon when making any investment decision. Certain information contained in here has been obtained from third-party sources. While taken from sources believed to be reliable, I have not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation. The content speaks only as of the date indicated.

Under no circumstances should any posts or other information provided on this website -- or on associated content distribution outlets -- be construed as an offer soliciting the purchase or sale of any security or interest in any pooled investment vehicle sponsored, discussed, or mentioned by a16z personnel. Nor should it be construed as an offer to provide investment advisory services; an offer to invest in an a16z-managed pooled investment vehicle will be made separately and only by means of the confidential offering documents of the specific pooled investment vehicles -- which should be read in their entirety, and only to those who, among other requirements, meet certain qualifications under federal securities laws. Such investors, defined as accredited investors and qualified purchasers, are generally deemed capable of evaluating the merits and risks of prospective investments and financial matters. There can be no assurances that a16z’s investment objectives will be achieved or investment strategies will be successful. Any investment in a vehicle managed by a16z involves a high degree of risk including the risk that the entire amount invested is lost. Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by a16z is available at Excluded from this list are investments for which the issuer has not provided permission for a16z to disclose publicly as well as unannounced investments in publicly traded digital assets. Past results of Andreessen Horowitz’s investments, pooled investment vehicles, or investment strategies are not necessarily indicative of future results. Please see for additional important information.