Blog

Half the money I spend on advertising is wasted; the trouble is I don’t know which half.

– John Wanamaker

Recently, Andrew Eifler has been thinking about the possibility of a bubble in digital advertising.

He raises a number of good points. In summary, they are:

1. People see unlimited growth potential for digital advertising. In fact, most internet-based companies launch intended to depend exclusively on advertising to support themselves.
2. Advertising is hard to value.
3. People are buying “new media advertising products” without a clear idea of how much they are worth.
4. Therefore, there is the chance of a bubble.

I’m going to agree with Mr. Eifler’s core conclusion, but disagree with a few of the assumptions he uses to get there.

First of all, advertising is inherently difficult to measure, as Mr. Eifler points out. You can do all the advertising you want, but the only metric you really know – or care about – is units sold.

There are two problems with this metric. The first is the (classic) issue of counterfactuals. Just because you sold x number of units with advertising, it doesn’t mean you wouldn’t have sold x number of units without advertising. Furthermore, just because sales declined by e units, you don’t know if the advertising was effective, or that without it units sold would have declined by 5e units.

Second, there’s the problem between long-term and short-term sales. A great deal of advertising is “informational” – you let the right people know that your product is available at a certain price. But more and more advertising is “aspirational” – attaching your product to a particular lifestyle. It’s done, crudely, in artistic or entertaining advertising; and a bit more subtly as product placement in TV shows.

The problem is how you measure the ROI of short-term and long-term advertising. For all you know, the ROI of your long-term advertising strategy could be negative – but how the hell would you detect that?

This problem of measurement is compounded by the proliferation of new advertising channels.

Now, I’d going to disagree with Mr. Eifler and say that the digital advertising is better than traditional forms of advertising, because you have (vastly) more metrics. In fact, you can even (sometimes) track an individual all the way from initially seeing an ad to purchase!

The additional metrics makes valuing easier, though I have to give a nod to Mr. Eifler and point out that it’s possible to “worship a false idol.” That is, you could construct an elaborate and mathematically beautiful valuing equation using all those metrics… that means precisely bugger all when it comes to ROI. The temptation of having metrics is using those metrics; but metrics are not reality, they are only a reflection of reality. They are subject to both measurement error and conceptual error – where what you think you are measuring is not, in fact, what you actually are measuring (the quintessential example is that IQ tests do not measure intelligence).

However, digital advertising leads to a larger problem – which is the proliferation of advertising channels.

Let’s say you spend $1m on TV advertising. Now, we can’t make many assumptions about how the effectiveness of that advertising will change as you raise spend to $10m, but it’s reasonable to assume that the relationship between spend and effectiveness is monotonic; that is, the effectiveness of your advertising campaign will not decrease if you double your spend.

However, the relationship could be anything from linear to (more likely some kind of S-Curve:

As you increase spending early on, advertising becomes more effective, per dollar. However, at some inflection point – in the middle of the graph above – advertising begins to become less effective per dollar. The effectiveness is still increasing; but each additional dollar gets you a little less than before.

(I’d appreciate it if some people in the advertising could check me on this wildly unfounded assumption).

Now, however, let’s say you spend $10 on a bunch of different channels – TV, radio, billboards, online banner ads, online search ads, sponsored viral videos, whatever.

How do you measure the effectiveness of not only each channel, but also all channels combined? This is not a trivial question; at the least, you need to consider 2ⁿ-1 factors, where n in the number of channels.. If you have TV and radio advertising, you need to consider (1) the effect of TV, (2) the effect of radio, and (3) the effect of radio and TV together.

If you bump that up to 3 channels – let’s say TV, radio, and billboards, then you have to consider (1) the effect of TV, (2) the effect of radio, (3) the effect of billboards, (4) the effect of TV and radio together, (5) the effect of radio and billboards, (6) the effect of TV and billboards, and (7) the effect of TV, radio, and billboards together.

To make matters worse, any significant interaction at the “higher” levels will invalidate a straightforward effect of any less complex combination. So if there is a 3-way interaction, then you cannot analyze TV independently; because if you say “TV works” that is conditional on both radio and billboards; you cannot then increase TV spend independent of radio and billboard spend and except to see it work more.

If you have 6 channels, let’s say, you’re looking at 63 different factors to understand.

To introduce another wrinkle – I know, sorry – the above assumes that everyone sees all advertising channels, i.e. that everyone comes from the same statistical population.

Except they don’t. Now, it would be simple if people who saw one channel didn’t see any of the other channels; but that isn’t true. Since you are not dealing with either the same population or independent populations, you need (somehow) to divide customers into “channel segments.” That is, people who see TV and billboard but not radio; people who see online ads and TV ads but not radio and billboards – etc.

This complexifies the problem, because you now need to figure out how each channel reacts to each combination.

Additionally, the above outlines a scenario where advertising is identical, e.g. increasing spend gets you more of the same. But not all advertising campaigns are made equal – the sheer creativity in the market means that the same dollar will get you a different product depending not only on which agency you hire, but also on your company. The agency may make a campaign which would be great for a company that is very similar but not the same as you. Or one that just slightly misses your target market, reducing its effectiveness.

So: the inputs are highly variable, the outputs are difficult to measure; even if you could measure them, interpreting them would be very close to impossible, even if you could do all that it’s still a counterfactual (you don’t know how close your conclusion is to the truth, since there’s no way to test it).

Therefore, accurately valuing advertising is pretty much impossible. You just have to be satisfied with the assumption of monotonicity, and vague claims that the interaction across channels is positive.

In the end, then, I agree with Mr. Eifler. But not because new media is hard to value; I think it’s easier to measure the post hoc effectiveness, which is a pretty good proxy. The digital scene may improve matters, due to additional metrics.

However, given the difficulty of valuing advertising and the expectation of unlimited growth, you certainly have the chance for a bubble.

The Wall Street Journal has a charming article on companies trying to fill gaps left by Google.

The company they feature is Quora, Inc. It’s founded by some very famous people – former CTO at Facebook – and was established in 2009. In March, some people invested at a pre-money valuation of $87.5 million (or: more than $10 million/employee, in a year. That’s some equity growth!). It’s particularly impressive since that was when the product was in private testing, before not even a business model but even traffic.

Quora had the bright idea to improve the Q&A space on the Internet. I mean, sure, there’s Yahoo! Answers; Metafilter and Amazon’s AskVille; Answerology, and a great deal more.

But, see, Quora has done something different:

The service allows people to pose or answer questions—working behind the scenes to route questions to the users who can best answer them.

Yes, it uses machine learning to route questions to capable users. This is a break from the current success-model, which is to treat it like a game – reward people with “points” of some kind.

And! Importantly! It’s a gap in Google’s product offerings! Google Answers was retired in 2006. Besides; Google Answers didn’t route questions to experts immediately – they had paid experts who had to trawl through the questions. It wasn’t scalable. Silly Google – no wonder they retired it!

Perhaps my sarcasm is come through a little strong.

Since 2007, Edward Chang – Director of Research, Google China – has been spearheading work in “classification and collaborative filtering.” What does that mean? Well, in 2007, Google launched Otvety (Russia) and Wenda (China) as trials for new applied of Google-researched technology in a product called “Knowledge Search.”

Edward Chang outlines this in a 2008 article outlining some of Google’s Data Management Projects:

Knowledge Search allows users to post questions and then matches experts to timely answer questions. The distinguishing feature of this product compared to competing products is that it offers online question classification, related-question recommendation, and topic-sensitive expert matching.

…. which sounds awfully similar to Quora. And it has the advantage of Google Researchers – experts in this small field of knowledge representation, question classification, and matching – working on it since 2007.

Given Quora’s lack of content, if Google simply ported the existing products over to the US, it would have a directly comparable product backed by the Google brand. And “directly comparable” is probably understating the technological advantage; you don’t put together something really sophisticated in a year, particularly if you’re busy trying to get a company together besides. Besides which, it’s not necessary; the marginal improvements in algorithms would be minuscule compared to spending their time building out the site.

So, if it wasn’t clear enough: I’m astonished at the market research which presented this as a “gap left by Google.” Yes, it certainly hasn’t been a priority; and yes, the current projects are research-orientated; and yes, it’s likely that it won’t become a priority. It’s entirely possible that Quora could become successful.

But that’s predicated on Google not doing anything with assets it already has. It’s certainly possible, and may even be rational – if Quora grows the market, then Google may decide to buy the product and (importantly) the people.

But beyond that, I’m astonished anyone would invest at a pre-money valuation of $87.5 million with this kind of situation, and absolutely no business model. And I quote:

Quora’s business model is currently unclear. Mr. D’Angelo, who became chief technology officer of Facebook in 2006 and left the company in 2008, said "if I had to guess, it would probably involve some kind of advertising at some point."

The defense, of course, is: “Find good people, let them do whatever; they’ll succeed.” And their team is, apparently, top-notch with a history of success.

I find it unconvincing.

via Mark Thoma, Claude Fisher has some interesting observations on crime in the United States:

Most notably:

Cities and neighborhoods that have received the largest influx of immigrants (including Mexican immigrants) have had — despite popular stereotypes to the contrary — the largest drops in criminal violence. (See, e.g., here and here.)

And in general, crime is declining:

Interesting stuff.

It also makes it difficult to explain why we have 25% of the world’s prison population in our borders, have a recidivism rate of 70-80%, and execute people by firing squad.