Evolution

This post is very confusing because it has nearly no structure; it is the jotting down of a few thoughts. Please be warned.

In my opinion, evolution is the most powerful framework for understanding the world – and what works. Evolution is the way the universe works.

I will go so far as to say: Anything that does not adhere to the precepts of evolution will either fail or stagnate.

Now, I don’t mean (just) biological evolution, the kind Charles Darwin wrote about. In fact, I know relatively little about the modern synthesis and molecular biology. Evolution, though, is not restricted to biology.

Evolution is simple. The entire theory can be explained with three words: Variation, Selection, Heredity. Make a change, see how well it works, and then do it again.

The change does not have to be random (and some modern evidence suggests that genetic change is not random), the selection criteria do not have to be immutable, and inheritance just means “repeat, because it worked.”

There is a tendency for people, such as (some) evolutionary psychologists, to attempt to reduce behavior to biology. This is not an evolutionary explanation; it is a reduction from one level of explanation to another level of explanation. They are very different things.

An increasingly popular approach in business is called “iteration.” While most noticeable in web startups (e.g. Basecamp), businesses everywhere are adopting it. The approach comes down to: Make something fast, show it to the customer, see what they think; fix problems, make more changes, see what they think.

Iteration is evolution by another name. Vary your product, measure it against the selection criteria (customers), then change what doesn’t work and improve what does.

On a side note, the concept of “iteration” is incomplete. It largely ignores the selection criteria – it takes that on faith. A more rigorous examination of the selection criteria makes obvious the similarities (and differences) between real customers, focus groups, surveys, etc. People should pay more attention to selection criteria.

The consequence of accepting evolution is that “designed” becomes a dirty word. Centrally managed, independently created, committee-verified… are all models bound to fail, compared with an evolutionary approach. Certainly, they may succeed; in fact, it’s inevitable that some will succeed. But success is far from inevitable.

If you want a system or a process to succeed, design it in an evolutionary fashion.

Economists like to claim that the market system is the most efficient allocation system. That may be true, given the highly idealized assumptions one must grant them before they will say that.

However, a market system is a two-dimensional reflection of an evolutionary system. Sellers in a market subject an array of similar but different goods to the selection criteria of buyers. Successful products are purchased in larger numbers; the adaptations which made them successful are then copied.

A traditional market analysis lacks the notion of hereditary, or of change between products over time. Once you add the notion of time in, the concept of a market is functionally identical to (but less flexible than) an evolutionary explanation of economics.

I’m obsessed with evolution, right now. I’m sure you can tell.

The biggest reason is that evolution is arational. It is not irrational, or against reason; it simply has nothing to do with reason. It functions independently of both truth and God. It is a system which progresses – changes, at the very least – without any component of the system (or the system itself) having (i) access to the truth, (ii) direction or intent, or (iii) awareness of the system. It changes endogenously, and requires very few assumptions to work.

It’s beautiful.

How to Value Advertising

Half the money I spend on advertising is wasted; the trouble is I don’t know which half.

- John Wanamaker

Recently, Andrew Eifler has been thinking about the possibility of a bubble in digital advertising.

He raises a number of good points. In summary, they are:

1. People see unlimited growth potential for digital advertising. In fact, most internet-based companies launch intended to depend exclusively on advertising to support themselves.
2. Advertising is hard to value.
3. People are buying “new media advertising products” without a clear idea of how much they are worth.
4. Therefore, there is the chance of a bubble.

I’m going to agree with Mr. Eifler’s core conclusion, but disagree with a few of the assumptions he uses to get there.

First of all, advertising is inherently difficult to measure, as Mr. Eifler points out. You can do all the advertising you want, but the only metric you really know – or care about – is units sold.

There are two problems with this metric. The first is the (classic) issue of counterfactuals. Just because you sold x number of units with advertising, it doesn’t mean you wouldn’t have sold x number of units without advertising. Furthermore, just because sales declined by e units, you don’t know if the advertising was effective, or that without it units sold would have declined by 5e units.

Second, there’s the problem between long-term and short-term sales. A great deal of advertising is “informational” – you let the right people know that your product is available at a certain price. But more and more advertising is “aspirational” – attaching your product to a particular lifestyle. It’s done, crudely, in artistic or entertaining advertising; and a bit more subtly as product placement in TV shows.

The problem is how you measure the ROI of short-term and long-term advertising. For all you know, the ROI of your long-term advertising strategy could be negative – but how the hell would you detect that?

This problem of measurement is compounded by the proliferation of new advertising channels.

Now, I’d going to disagree with Mr. Eifler and say that the digital advertising is better than traditional forms of advertising, because you have (vastly) more metrics. In fact, you can even (sometimes) track an individual all the way from initially seeing an ad to purchase!

The additional metrics makes valuing easier, though I have to give a nod to Mr. Eifler and point out that it’s possible to “worship a false idol.” That is, you could construct an elaborate and mathematically beautiful valuing equation using all those metrics… that means precisely bugger all when it comes to ROI. The temptation of having metrics is using those metrics; but metrics are not reality, they are only a reflection of reality. They are subject to both measurement error and conceptual error – where what you think you are measuring is not, in fact, what you actually are measuring (the quintessential example is that IQ tests do not measure intelligence).

However, digital advertising leads to a larger problem – which is the proliferation of advertising channels.

Let’s say you spend $1m on TV advertising. Now, we can’t make many assumptions about how the effectiveness of that advertising will change as you raise spend to $10m, but it’s reasonable to assume that the relationship between spend and effectiveness is monotonic; that is, the effectiveness of your advertising campaign will not decrease if you double your spend.

However, the relationship could be anything from linear to (more likely some kind of S-Curve:

As you increase spending early on, advertising becomes more effective, per dollar. However, at some inflection point – in the middle of the graph above – advertising begins to become less effective per dollar. The effectiveness is still increasing; but each additional dollar gets you a little less than before.

(I’d appreciate it if some people in the advertising could check me on this wildly unfounded assumption).

Now, however, let’s say you spend $10 on a bunch of different channels – TV, radio, billboards, online banner ads, online search ads, sponsored viral videos, whatever.

How do you measure the effectiveness of not only each channel, but also all channels combined? This is not a trivial question; at the least, you need to consider 2ⁿ-1 factors, where n in the number of channels.. If you have TV and radio advertising, you need to consider (1) the effect of TV, (2) the effect of radio, and (3) the effect of radio and TV together.

If you bump that up to 3 channels – let’s say TV, radio, and billboards, then you have to consider (1) the effect of TV, (2) the effect of radio, (3) the effect of billboards, (4) the effect of TV and radio together, (5) the effect of radio and billboards, (6) the effect of TV and billboards, and (7) the effect of TV, radio, and billboards together.

To make matters worse, any significant interaction at the “higher” levels will invalidate a straightforward effect of any less complex combination. So if there is a 3-way interaction, then you cannot analyze TV independently; because if you say “TV works” that is conditional on both radio and billboards; you cannot then increase TV spend independent of radio and billboard spend and except to see it work more.

If you have 6 channels, let’s say, you’re looking at 63 different factors to understand.

To introduce another wrinkle – I know, sorry – the above assumes that everyone sees all advertising channels, i.e. that everyone comes from the same statistical population.

Except they don’t. Now, it would be simple if people who saw one channel didn’t see any of the other channels; but that isn’t true. Since you are not dealing with either the same population or independent populations, you need (somehow) to divide customers into “channel segments.” That is, people who see TV and billboard but not radio; people who see online ads and TV ads but not radio and billboards – etc.

This complexifies the problem, because you now need to figure out how each channel reacts to each combination.

Additionally, the above outlines a scenario where advertising is identical, e.g. increasing spend gets you more of the same. But not all advertising campaigns are made equal – the sheer creativity in the market means that the same dollar will get you a different product depending not only on which agency you hire, but also on your company. The agency may make a campaign which would be great for a company that is very similar but not the same as you. Or one that just slightly misses your target market, reducing its effectiveness.

So: the inputs are highly variable, the outputs are difficult to measure; even if you could measure them, interpreting them would be very close to impossible, even if you could do all that it’s still a counterfactual (you don’t know how close your conclusion is to the truth, since there’s no way to test it).

Therefore, accurately valuing advertising is pretty much impossible. You just have to be satisfied with the assumption of monotonicity, and vague claims that the interaction across channels is positive.

In the end, then, I agree with Mr. Eifler. But not because new media is hard to value; I think it’s easier to measure the post hoc effectiveness, which is a pretty good proxy. The digital scene may improve matters, due to additional metrics.

However, given the difficulty of valuing advertising and the expectation of unlimited growth, you certainly have the chance for a bubble.

Racing to Fill Google’s (Non-Existent) Gaps

The Wall Street Journal has a charming article on companies trying to fill gaps left by Google.

The company they feature is Quora, Inc. It’s founded by some very famous people – former CTO at Facebook – and was established in 2009. In March, some people invested at a pre-money valuation of $87.5 million (or: more than $10 million/employee, in a year. That’s some equity growth!). It’s particularly impressive since that was when the product was in private testing, before not even a business model but even traffic.

Quora had the bright idea to improve the Q&A space on the Internet. I mean, sure, there’s Yahoo! Answers; Metafilter and Amazon’s AskVille; Answerology, and a great deal more.

But, see, Quora has done something different:

The service allows people to pose or answer questions—working behind the scenes to route questions to the users who can best answer them.

Yes, it uses machine learning to route questions to capable users. This is a break from the current success-model, which is to treat it like a game – reward people with “points” of some kind.

And! Importantly! It’s a gap in Google’s product offerings! Google Answers was retired in 2006. Besides; Google Answers didn’t route questions to experts immediately – they had paid experts who had to trawl through the questions. It wasn’t scalable. Silly Google – no wonder they retired it!

Perhaps my sarcasm is come through a little strong.

Since 2007, Edward Chang – Director of Research, Google China – has been spearheading work in “classification and collaborative filtering.” What does that mean? Well, in 2007, Google launched Otvety (Russia) and Wenda (China) as trials for new applied of Google-researched technology in a product called “Knowledge Search.”

Edward Chang outlines this in a 2008 article outlining some of Google’s Data Management Projects:

Knowledge Search allows users to post questions and then matches experts to timely answer questions. The distinguishing feature of this product compared to competing products is that it offers online question classification, related-question recommendation, and topic-sensitive expert matching.

…. which sounds awfully similar to Quora. And it has the advantage of Google Researchers – experts in this small field of knowledge representation, question classification, and matching – working on it since 2007.

Given Quora’s lack of content, if Google simply ported the existing products over to the US, it would have a directly comparable product backed by the Google brand. And “directly comparable” is probably understating the technological advantage; you don’t put together something really sophisticated in a year, particularly if you’re busy trying to get a company together besides. Besides which, it’s not necessary; the marginal improvements in algorithms would be minuscule compared to spending their time building out the site.

So, if it wasn’t clear enough: I’m astonished at the market research which presented this as a “gap left by Google.” Yes, it certainly hasn’t been a priority; and yes, the current projects are research-orientated; and yes, it’s likely that it won’t become a priority. It’s entirely possible that Quora could become successful.

But that’s predicated on Google not doing anything with assets it already has. It’s certainly possible, and may even be rational – if Quora grows the market, then Google may decide to buy the product and (importantly) the people.

But beyond that, I’m astonished anyone would invest at a pre-money valuation of $87.5 million with this kind of situation, and absolutely no business model. And I quote:

Quora’s business model is currently unclear. Mr. D’Angelo, who became chief technology officer of Facebook in 2006 and left the company in 2008, said "if I had to guess, it would probably involve some kind of advertising at some point."

The defense, of course, is: “Find good people, let them do whatever; they’ll succeed.” And their team is, apparently, top-notch with a history of success.

I find it unconvincing.