In 1971 Edward de Bono published Practical Thinking, and has revised it multiple times; the last time in 1992.

It’s a charming little book, largely because – despite making some false statements – his advice is excellent, practical, and should improve thinking for almost anyone who reads (and applies) it.

The most interesting parts of the book, to me – given I have just completed a major in epistemology, or the study of knowledge (sort-of “how to think”) – was the advice about certainty. It’s well-known (now) that the feeling of certainty people sometimes have is bonkers. de Bono breaks down why it’s bonkers; but also provides ways of avoiding the issue.

I’m not going to re-hash his book, in part because he provides an excellent summary at the back of the book you can reference (and, really, it’s $4).

But the most important takeaway for managers and other “practical thinkers” is the de Bono’s discussion on the tyranny of the YES/NO system. It’s a simple insight: If you keep saying “No” to new ideas, the idea you end up with will be the first idea whose answer isn’t “No.” That is, it will not be the best idea; it will be the first mediocre idea. Abandoning the “YES/NO” system of brainstorming is really rather important.

If you’ve studied, oh, logic, Quine, cognitive psychology, and the philosophy of science, all this stuff will be old hat (and some of it wrong). If not,  I highly recommend it.

Today, I read The Science of Fear (2008) by Daniel Gardner. It’s a remarkably well-done book for what it is – namely, a journalist’s (informed) overview of some of the psychological components of fear, and a large number of example as to how people exploit that tendency to fear.

It’s a nice book because he relies on rather well-accepted psychological research, while going into great depth on examples. It helps people to understand that these principles actually mean something.

On the other hand, for those people who were looking for a explanation about how fear works inside the brain – such as myself – it’s a bit lacking. And the reliance on solid psychological principles means I didn’t learn any new psychology. Regardless, I enjoyed the read because it was well-written and interesting.

In lieu of a review, allow me to review some of his points. In the course of the book, Mr. Gardner outlined three ways the brain screws up, leading people to irrational fears.

1. The Availability Heuristic

The availability heuristic is a pretty good rule. It’s a general cognitive bias – pretty robust across all humans – and works out to people predicting the probability of events in proportion to how many instances of it they can recall (are available).

If you do something a lot – work on computers, go hunting, etc – then over time you establish a battery of experiences. If someone asked you how probable something was, you could reach back into your experience and get a feel for how many times you’ve seen it – and give a pretty good example.

The big advantage is that it’s computationally very fast. If you need to make a split-second decision, you want it to be fast.

It’s also extensible: that is, people don’t differentiate between their own experiences (memories) and other people’s (stories). This works out really well if you talk to people who do the same thing as you do – say, a bunch of hunters sitting around a fire swapping stories. That way, you can tap into the knowledge of your entire community (if you haven’t yet experienced it, you don’t know how common it is – hearing stories of experiences can both ameliorate your ignorance and give you ideas of what to do to deal with it).

But therein lies the rub. The media specialize in providing stories – really compelling anecdotes – about things that happen. The brain doesn’t differentiate based on sources, so the availability heuristic can be screwed in the incorrect direction. People vastly overestimate the risk of terrorism, kidnappings, and murder; but vastly underestimate the risk of car accidents, drowning, diabetes, etc.

2. Confirmation Bias

Confirmation bias is an old favorite of psychologists, simply because it explains so much.

It’s pretty simple, actually. Once you believe something, your brain tends to look for other instances of it – confirming instances. It does not, however, look for falsifying instances for your belief. Sometimes, your brain will even change it’s recollection of the facts to conform to your current belief (one example is the “rose-colored glasses” effect; you believe the past was better, so you unconsciously modify your memories of the past to make it match your belief).

But it also means once you believe something about, say, terrorism, you’ll focus on the positive (that is, supporting) instances – and ignore the others. No terrorist attacks does not affect your belief about the danger of terrorism even though a terrorist attack down – which is illogical. It’s a binary outcome, therefore one outcome value  should be just a good a predictor as the other. The brain doesn’t think so.

A rather insidious effect of confirmation bias concerns the use of statistics. If you believe someone, and you come across a statistic (or a story) you disagree with, you’re going to scrutinize it very closely. If, however, you come across a statistic which supports your belief – then, hey, no need to question the source or the methodology, it’s obviously correct. People apply different levels of evaluation to information that conforms with their existing beliefs to information that violates their existing beliefs.

3. The Urge to Conform

Conformity has been studied a great deal, and the results are pretty consistent. When people are in a group and a task is difficult, you see more conformity. That is, lower confidence in the result for any one individual means that people are more willing to accept a group consensus. Funnily enough, though, each individual’s belief in the accuracy and reliability of the group consensus goes way up – even though the confidence of any individual’s conclusion is low.

Mr. Gardner makes the important point that conformity actually serves a good purpose. If you’re on the African plains, and everyone around you begins to get worried about a tiger in the grass – well, even if you can’t see the tiger yourself, there’s a pretty good reason to take precautions. More formally, it allows all members of a group to take advantage of the knowledge from all members of the group, and not rely on their own knowledge all the time.

The problem is that once a belief has taken hold in the general population, it’s bloody hard to get rid of. The combination of conformity – people fall into line – and the confirmation bias means that as a group, people don’t deal with falsifying evidence well at all. Mr. Gardner goes through a hilarious number of examples showing that (i) people say they believe something because of the evidence, (ii) you prove the evidence is wrong, (iii) people still believe it despite accepting that the evidence is wrong.

A Passing Note

In addition to those three psychological features, Mr. Gardner notes a few other issues. Here’s one I found striking.

It has to do with pointing out how badly people deal with numbers. People have no innate ability to deal with numerical data; though they do have a pretty good ability to deal with proportions. Unfortunately, this isn’t a good thing.

Mr. Gardner gives a great example. Take two groups of people: in both, tell them they are reviewing how much money to devote to improving airport safety. Tell the first group that implementing the precautions will save 150 lives; tell the second that it will save 98% of 150 lives. Consistently, people rate saving 98% of 150 lives higher than 150 lives (that is, the second group would devote more money to the project then the first group, even though they were saving objectively fewer people).

And don’t get started on how bad people are with probability – it doesn’t bear thinking about.

A Brief Conclusion

The Science of Fear rests on some good psychology, and goes into a large number of examples as to how human reason fails us when it comes to knowing what to fear.

The real effect of the book is to persuade people to be less afraid; it reduces fear. Mr. Gardner systematically goes through most hot-button political issues, and shows how the data doesn’t back up the fear-mongering. Not only is he persuasive, but he writes in such a fashion that you’ll pick up an innate skepticism of the media (if you didn’t already have it) and a deeper skepticism for anecdotes (if you have no statistical background).

It’s certainly worth the time just for that.

Taylor Buley, writing on the Velocity blog at Forbes, has the provocative title of “Google Isn’t Just Reading Your Links, It’s Now Running Your Code.” Mr. Buley goes onto explain that “for years it’s been unclear whether or not the Googlebot actually understood what it was looking at or whether it was merely doing "’dumb’ searches for well-understood data structured like hyperlinks.” In other word, Google has built a Javascript interpreter!

The source for this headline comes directly from Google:

On Friday, a Google spokesperson confirmed to Forbes that Google does indeed go beyond mere "parsing" of JavaScript. "Google can parse and understand some JavaScript," said the spokesperson.

So it’s confirmed, then.

Mr. Buler spends most of his article explaining that building a Javascript parser is really fucking hard. In fact, a quote from one of his experts isolates the key problem – how long the code will run – and says that “The halting problem is undecidable," There is no algorithm that can solve it. Well, OK, I suppose, but couldn’t you process a lot and cut it off at an arbitrary point? Sure you’d miss some stuff, but surely you’d get enough?

Actually, that’s what another expert says:

"It’s hard to analyze a program using another program," the person says. "Executing [JavaScript code] is pretty much that’s the only way they can do it."

Mr. Buler believes this is a great accomplishment, and quite unknown.

He’s right on one count.

In a previous post, I cited a paper “Data Management Projects at Google” and talked about Edward Chang. Well, the paper is actually about three projects, and one of those is “Indexing the Deep Web,” spearheaded by Jayan Madhavan. In that 2008 paper, Dr. Madhavan had this to say about Javascript:

While our surfacing approach has generated considerable
traffic, there remains a large number of forms that continue
to present a significant challenge to automatic analysis. For
example, many forms invoke Javascript events in onselect
and onsubmit tags that enable the execution of arbitrary
Javascript code, a stumbling block to automatic analysis.
Further, many forms involve inter-related inputs and accessing
the sites involve correctly (and automatically) identifying
their underlying dependencies. Addressing these and
other such challenges efficiently on the scale of millions is
part of our continuing effort to make the contents of the
Deep Web more accessible to search engine users

It would seem they solved this problem! (This is a big accomplishment). When did they solve it? Recently?

Well, sort of. In a 2009 paper called “Harnessing the Deep Web: Past, Present, and Future.” In it, they say this:

We note that the canonical example of correlated inputs,
namely, a pair of inputs that specify the make and model of
cars (where the make restricts the possible models) is typically
handled in a form by Javascript. Hence, by adding a
Javascript emulator to the analysis of forms, one can identify
such correlations easily.

So let’s back up.

What is Google going? They’re accessing structured data hidden behind form submissions. Now, we say the information is “hidden” behind form submissions because you have to submit the form to get the data. One approach – the ”dumb” approach – is to generate all possible result URLs and then crawl all of them.

But. Those clever folks at Google noticed this might be a problem:

For example, the search form on cars.com has 5 inputs and a Cartesian product will yield over 200 million URLs, even though cars.com has only 650,000 cars on sale.

The challenge, then, is making fewer URLs. Thus, they intelligently developed an algorithm with this property:

We have found that the number of URLs our algorithms generate is proportional to the size of the underlying database, rather than the number of possible queries.

How do they do this? Well, one big challenge is (as noted above) the inputs in one field can depend on the inputs in another field. Google has taken to constructing databases of “interrelated data” (like manufacturer and car model) so they can automatically detect the data the form wants and limit their indexing accordingly.

But to detect when some fields on a form are interrelated, you… need to have more than the HTML. In fact, almost all input-dependent forms rely on Javascript to change the values around after a selection.

Well, the clever researchers at Google knew they needed to determine which fields in a form were interrelated. They also figured that they only needed to determine this once, because once they knew which fields were related, they could automatically generate their URLs using their generation algorithms.

As you can imagine, if you only need to do it once (for each form), then it becomes practical to emulate. You emulate one form, and get 650,000 URLs to index with solid data. It’s cheap – so cheap, it’s almost worth getting a human to do it. (Except no Googler would think of that!).

But – and here’s the thing – to emulate the behavior of a form driven by Javascript you have to have the Javascript files. You need to download them, and then execute them.

In other words, the second expert Mr. Buley consulted is spot-on. Google is executing the Javascript code to find out something very specific (which fields on a form are interrelated, and presumably anything done in an onsubmit event that would alter the indexing URL).

This is not news. It’s publically available information – very easily, though Google Scholar, and even easier if you’re following Google’s main researchers – and there is no reason to resort to speculation to answer the question. They’ve been accessing the Deep Web – the web hidden behind forms – for years; Javascript is an obvious stumbling block; Google researchers have papers published on it (frequently presented at conferences!).

It is galling to see a reporter say that something is “unclear” when it is very difficult to make something clearer. In 2008, Jayant Madhavan wrote on the Google Webmaster Central blog talks about crawling through forms to get to the Deep Web – this stuff isn’t restricted to academic papers easily accessible through Google Scholar and surfaced in regular Google results. No, it’s even in the blogosphere.

I think I’ve gone a bit too far, so I’ll stop now.

David Brooks has a column today on the debacle created by the Rolling Stones report on General Stanley McChrystal. Ignoring the politics (why did the Rolling Stones reporter adopt that specific narrative, which he knew would result in political controversy?), there are a couple of points Mr. Brooks raises which I think are worth addressing.

The Psychology of Groups

First, he points out that it’s natural for people in small groups to complain about people on other groups as a way to relieve stress and build a sense of community. He is quite right about this; contemporary research in social psychology has demonstrated, time and again, that people (i) believe that their group is better, and (ii) have less awareness of people in other groups as people.

Possibly the best book on the subject, The Psychology of Stereotyping by David Schneider, goes into this in depth (I highly recommend the book). To summarize,

the “Tajfel effect” occurs when people have ingroup bias for no reason. The most obvious example in the real world is the attachment people develop to sports team from the cities they live in. Why do people associate so strongly with a certain team just because they live nearby? The most striking psychological example is the “minimal group” situation. If you take a set of people and divide them into two groups in a completely arbitrary fashion – say, flipping a coin – then even if you tell them ahead of time how the groups were divided, people show ingroup bias. That is, if you survey people they believe that their group is on average smarter, more attractive, etc. (This effect is robust, i.e. it holds true if you measure it in a different way – say, eliminating surveys).

The second effect is that people in the group will try to maximize ingroup differentiation (and minimize outgroup differentiation – e.g. “Those Muslims are all the same”). It’s a way, in essence, of “dehumanizing” people who are different from you. Why? Well, the easy answer is that everyone has limited cognitive power. You take shortcuts and, in general, while it’s very important to know how people in your own groups are different from each other, it’s essentially irrelevant to know the same information for people you don’t identify with. All you really need to know are attributes (or stereotypes) like “snakes tend to be dangerous” or “maggots are bad.” Applied to people, it can be “jocks tend to be dumb” or “politicians tend to lie.” Limited interactions mean you don’t need to know any more…. you just need to know enough to deal with them if you encounter them.

Thus, it’s not surprising that Genera McChrystal’s group exhibited (as Rolling Stones reported) “arrogance” about their own capabilities and denigrated people who (i) they didn’t deal with much, and (ii) whom they had nothing in common with. If you have something in common with someone, then you are by definition part of some group – and even if it’s a weak tie, the same ingroup/outgroup bias comes into play (just weaker, obviously).

In fact, it’s important to note that the fact that General McChrystal’s team exhibited such behavior. It demonstrates that he had a unified group. The Rolling Stones report really shows a healthy team. Why? Because the General was taking people from ostensibly different groups (computer geeks from MIT, special ops, soldiers, etc) and fusing them into a unified team. It’s very important to note that they complained about other people outside of the group. It would be very easy, in contrast, for the special ops guys to whinge about the computer geeks, or soldiers, etc. They didn’t: they complained about people outside the group.

That is, in fact, an indication of General McChrystal’s “greatness,” and is something – as Rolling Stones noted – which has given him such a reputation: the ability to pull people from multiple different backgrounds and construct a highly functional team.

The Cult of Personality

The second point David Brooks makes is encapsulated by this paragraph:

Then, after Vietnam, an ethos of exposure swept the culture. The assumption among many journalists was that the establishment may seem upstanding, but there is a secret corruption deep down. It became the task of journalism to expose the underbelly of public life, to hunt for impurity, assuming that the dark hidden lives of public officials were more important than the official performances.

I can’t really disagree with him: reading politics or watching the news is no longer about policies is about the personal lives of the politicians. “Would we like this guy if we met him in a bar?” or “Do we want to imitate this guy?” when, really, such concerns are patently irrelevant to the quality of the job they do.

Popular success is not determined on results; it is determined by personality.

Politicians have become celebrities. They establish a personal brand, and suck people into believe that they are a certain “type of person.”

It’s an interesting extension of the representative democracy model. We elect politicians who make political judgments for us – that is, they represent our collective interests, ideally. With the cult of personality, we elect people who seem like “our kind of people” and trust them to make the kind of decisions we’d like them to make. Thus, instead of judging a politician on how well they represented our interests, we judge them on whether or not we still feel like we have something in common with them.

Politicians as a proxy for people; personality as a proxy for ideology.

Of course, the sad thing is that personality is a terribly proxy for either ideology or effectiveness.

It’s also a classic lesson in Peter Drucker’s wisdom that “You can’t manage what you don’t measure” and “what’s measured improved.”

And as politicians learn how important their personality “brand” is, they become obsessed on maintaining that brand. But they found – particularly with the rise of TV – that managing their personality brand has very little to do with passing legislation. On the other hand, it has a great deal to do with (i) sound-bytes, and (ii) relationships with other politicians, and (iii) endorsements by famous people (and other politicians).

As such, the “personality brand” of politicians improves. But everything extraneous to that – e.g. actually reading the laws they’re voting on – deteriorates, because it has no impact on what’s being measured.

In fact, personality has become so important that people think the results (which personality is acting as a proxy for) is irrelevant.

Such is the fate of General McChrystal. He is – according to many on both sides of the political divide – highly capable, has proven success in multiple arenas, and is showing success in Afghanistan. Yet his personality has been judged lacking, due to the highly filtered view of it (and his team) provided by the Rolling Stones reporter.

Mr. Brooks is correct when he calls the “exposure ethos” damaging. But he misdiagnosis the problem – it’s not exposure, per say, it’s exposure of the wrong things. Actions are something we should care about – such as Nixon’s ethical abuses. Not “kvetching” as Mr. Brooks calls it; healthy team dynamics (that should be kept within the team).

Specifically, it’s a reply to Andrew Eifler who posted the blog post I responded to. He raised this point:

On the subject of variables and, as you point out, there can be quite a few – i think one of the biggest issues is how we quantify presence on each media channel. Universally the units that are used are “GRPs” or Gross Rating Points which are the product of “Reach” and “Frequency” against your target audience. For advertising measurement to really progress we really need a new unit of measurement. The system of GRPs worked great when the only media options were TV, Print, and Radio – but in today’s world, with such a fragmented media landscape, there really needs to be a more fitting measure. Maybe something like “Persuasion units?” Interested to hear what you think about this.

- Andrew Eifler

In general, I doubt I could come up with a decent replacement statistic, simply because the data is so poor. I agree, however, that the current statistics you use – GRPs, and also TRPs – are woefully bad.

Why GRPs Are Bad

The statistic Gross Rating Points (GRPs) is calculated by multiplying percentage reach by frequency (is that average frequency?). Now, this is all well and good if all you’re interested in is "impressions" as in "banner ad impressions." But the experience of NON-obnoxious (overlay) and NON-personalized banner ads should lead people to be VERY skeptical of the worth of impressions as something useful.

Click-through rates on banner ads are what, 0.2-0.3%? If "actions" (like clicking through is an action) on TV ads are similar – I wouldn’t be surprised – that’s very low. And what’s the conversion rate after that? 5%? 10%? Depressing, but I suppose it’s beside the point.

A more important observation is that (i) people will have variable marginal returns to seeing an ad repeatedly, and (ii) the distribution of view frequencies are highly unlikely to be normally distributed through the population or segment. In the case of (i), I think that the marginal returns are likely to resemble a S-curve as well; of course if your ad is particularly irritating, it may tend negative after some additional inflection point.

I hope the current methodology takes (ii) into the account; I would expect some members of the population to be more likely to see an ad more times. I suppose you can mitigate this by segmenting the population in the right way (e.g. segment by number of times they have seen/are likely to see and/or respond to the advertising). Otherwise, you’re asking to be mislead.

The Problem of Advertising Statistics

However, there’s a deeper problem with such “industry standard” statistics that do not measure the result they measure the deliverable. Or: they do not measure what the customer cares about (increased sales); they measure what the advertising did. “We showed most of the people you care about this ad 3 times – and surveys (?) indicate that consumers remember the ad!”

Sure, I get that’s the best you do. You’re not selling increased sales; you’re selling something very specific. If the company gets increased sales, good for them; if not, well they decided to do it in the first place. Hell, figuring out how to grow sales is (allegedly) why company executives are paid so much.

But, it would seem to me, part of empower companies to make their own choices about how much to spend is giving the information they care about – “How much did the advertising campaign increase my revenue and profit?” – not the deliverables (which are only really of interest internally to the advertising company). Making the process transparent – letting the company buying advertising know what advertising was delivering to their market – may (i) seem like a good thing to do, and (ii) help justify fees to the customer, but it’s really not a very bright idea.

An Unfounded Extrapolation

The reason is simple: it reduces the profit margins of marketing companies. Oh, not in the short term – but in the long term. The selection pressures for marketing shift from making the most effective marketing to making the… well, less effective marketing. If you can make the process less effective, you get paid more. At best, efficiency will stop increasing.

The approach should be reversed. Marketing companies shouldn’t sell different product lines – “Yes, you can spend $3m on TV, $1.5m on radio, $1m on billboards, and $2.5m on online advertising, we can do that for you” – they should be selling increases in sales. Hell, if you really wanted to motivate an advertising company you’d get them some percentage share of the increased revenue attributed to advertising (though with provisions to prevent gaming).

Sure, yes, I know I’m both reaching a bit and have no empirical evidence to make such an allegation and information about how (digital) advertising is changing things implies the opposite. The amount of innovation occurring in digital advertising (albeit, sometimes creepy innovation) is staggering. The allocation algorithms behind Google’s AdWords and AdSense programs are mathematically guaranteed to lead to the most optimal outcome for all involved; the personalization possible with customer tracking (e.g. DoubleClick) is getting there; the shift to Actions and not Impressions is coming fast, etc. I just don’t like statistics like the GRP (unless I’ve completely misunderstood it…).

Screw the Literature

Of course, it’s not like the (academic) literature is any better. I dipped into a couple of (mathematical) marketing journals earlier today when I was researching the (earlier) response; I lost most of my references, though, and as this is not an academic paper I’ll refrain from re-locating them. The upshot, though, is that modern mathematical and economic accounts of advertising assume that (i) you can segment your population well, and (ii) that all segments are homogenous (note that (ii) implies (i)). Is this accurate? – I would be terribly surprised if the segmenting was that accurate.

Given these constraints, most models assume that (i) you represent advertising as contributing to some sense of “goodwill” among people, and (ii) in the absence of advertising the amount of goodwill a customer feels towards you or your product declines (which is a great way to justify advertising! –> apparently it’s a well-established empirical pattern; I’d like to check that).

This is known as “inventing a variable which doesn’t really exist.” The scientific justification for it is that “goodwill hypothetically relates to several variables – known and “latent” (which means “unobservable”) – all of which are correlated, such as goodwill in an “index” for those other variables.” Actually, what they would probably tell you is that goodwill is a latent variable that can be distilled via structural equation modeling from some empirically observable variables – but it’s a distinction without a difference. Mostly.

Now, I suppose goodwill is a better thing to try and measure than GRP. However, it’s still (i) artificial, and (ii) has nothing to do with revenue or profit. Thus, the literature is little better.

However, one good point (which I had not considered) is that given these assumptions, then the approach you take is influenced by whether or not you are trying to maximize goodwill at some point t, or if you are trying to maximize the integral of goodwill over the advertising campaign. The obvious example of the former is in selling tickets to some event.

Closing Thoughts

Measuring advertising is hard. So the tools you have are limited.

However, I think the statistics cited should have more to do with what the company buying advertising needs. That may be revenue and profit, of course – but it may be something else. I mentioned aspirational advertising in the last post; I have no idea how to measure that (except really crudely, with surveys and interviews).

And I think the statistics used should have less to do with impressions, unless you’re trying to improve effectiveness (“Our GRP is 250, but sales only increase 0.4%! Something needs to change!”). It’s certainly never something you should show the customer.

Statistics related to inputs are useless. GRP, and similar statistics, look pretty much at what you put into the campaign – just like college rankings look at what goes into the colleges (SAT scores, money, etc). They don’t measure outputs, e.g. how successful each college student is, or how much advertising increased sales. Why? Because it’s hard to measure.

But abandoning something just because it’s hard is no way to live; and adopting an inputs-based measurement process will do nothing but increase cost (like it’s done for the college industry).

Unless, of course, that’s what you want.

Also, I have to confess that the reply to this I wrote earlier was lost when my computer crashed… teach me to use something lacking autosave.

The core points the author makes are:

1. Everyone embellishes a little (and if you don’t, you’re at a disadvantage of those who do – the tyranny of asymmetric information!).
2. People don’t necessarily do it deliberately. Memory is unreliable, and there are different norms in different locations/industries/companies.
3. Employers really don’t like it.
4. It’s becoming a lot easier to both record and find (Google) transgressions.

The overall effect of the article is to discourage the embellishment of statements that are easy to check, and nudge people toward those which are “minor.” By minor, the author seems to mean “not binary;” that is, you either did or didn’t get a degree; you either did or didn’t hold such a position, etc.

The rest of the post is divided into two sections: they have nothing to do with each other. I just had two reactions to the piece.

Personal Reaction

My take is that asking people to accurately represent their experiences is like asking money to rain from the sky. Memory just doesn’t work that way, and cognitive errors (such as cognitive dissonance) ensure that people’s memories will always be self-serving.

Now, I don’t like to embellish, personally, because I fell like I’m lying. If anything, I tend to understate what I’ve done and expect the other person to know the unstated difficulties that needed to be overcome (an OK strategy when talking to experts in an area – I blame my years of academia, where people actually knew what I was leaving out).

But in general, and when looking for a job in particular, either understating one’s accomplishments or telling the absolute truth is more deceptive that embellishing a bit. If a hiring manager expects the interviewee to embellish, then they’ll discount all statements, since they have no idea which are embellished and which are not. (Also because a false negative is better than a false positive – better to drive away someone qualified than to hire someone incompetent).

As a result, considerate advice to the job-seeker must be to embellish a bit, but not too much… just around as much as everyone else is.

The Bigger Question

Now, there’s a bigger question the article raises. To what extent do these embellishments matter? Or, worse, qualifications at all?

Take Marilee Jones. The article says this:

…a former dean of admissions at the Massachusetts Institute of Technology and the author of a popular guide to the college admissions process. Although she encouraged college applicants not to overstate their accomplishments, Jones resigned from her position in 2007 after it was discovered that she had fabricated two academic degrees on her initial job application in 1979 and added a third later on.

Wikipedia reveals that (i) her book received terribly critical acclaim, being featured on the NYT, WSJ, etc, and (ii) in 2001 she received MIT’s Excellence Award for Leading Change.

In other words, Marilee Jones was a highly capable and highly effective employee.

Not having – and even lying about – her degrees did not detract from that.

Another example that springs to mind is former RadioShack CEO David Edmondson. Mr. Edmondson lied about his college degree, as well, but was also highly capable and added a tremendous amount of value to RadioShack (hired in 1994, won multiple industry awards, promoted internally to CEO). Apparently the trade rag Advertising Age recognized him as “one of 100 top marketers.”

These two examples have something in common:

They lied about their college degrees. Perhaps they lied about other things, but…

… that raises the question of whether or not college degrees made any different to one’s competence at work? Are those four years wasted?

It’s a scary question, as someone who’s just graduated (and currently unemployed!); and a serious issue for an industry whose pricetag has skyrocketed for decades.

(My answer deserves its own post, but: Sort-of. Employers use college as a way of filtering applications so they don’t have to determine the real qualifications… it’s a proxy for competence).

In my opinion, evolution is the most powerful framework for understanding the world – and what works. Evolution is the way the universe works.

I will go so far as to say: Anything that does not adhere to the precepts of evolution will either fail or stagnate.

Now, I don’t mean (just) biological evolution, the kind Charles Darwin wrote about. In fact, I know relatively little about the modern synthesis and molecular biology. Evolution, though, is not restricted to biology.

Evolution is simple. The entire theory can be explained with three words: Variation, Selection, Heredity. Make a change, see how well it works, and then do it again.

The change does not have to be random (and some modern evidence suggests that genetic change is not random), the selection criteria do not have to be immutable, and inheritance just means “repeat, because it worked.”

There is a tendency for people, such as (some) evolutionary psychologists, to attempt to reduce behavior to biology. This is not an evolutionary explanation; it is a reduction from one level of explanation to another level of explanation. They are very different things.

An increasingly popular approach in business is called “iteration.” While most noticeable in web startups (e.g. Basecamp), businesses everywhere are adopting it. The approach comes down to: Make something fast, show it to the customer, see what they think; fix problems, make more changes, see what they think.

Iteration is evolution by another name. Vary your product, measure it against the selection criteria (customers), then change what doesn’t work and improve what does.

On a side note, the concept of “iteration” is incomplete. It largely ignores the selection criteria – it takes that on faith. A more rigorous examination of the selection criteria makes obvious the similarities (and differences) between real customers, focus groups, surveys, etc. People should pay more attention to selection criteria.

The consequence of accepting evolution is that “designed” becomes a dirty word. Centrally managed, independently created, committee-verified… are all models bound to fail, compared with an evolutionary approach. Certainly, they may succeed; in fact, it’s inevitable that some will succeed. But success is far from inevitable.

If you want a system or a process to succeed, design it in an evolutionary fashion.

Economists like to claim that the market system is the most efficient allocation system. That may be true, given the highly idealized assumptions one must grant them before they will say that.

However, a market system is a two-dimensional reflection of an evolutionary system. Sellers in a market subject an array of similar but different goods to the selection criteria of buyers. Successful products are purchased in larger numbers; the adaptations which made them successful are then copied.

A traditional market analysis lacks the notion of hereditary, or of change between products over time. Once you add the notion of time in, the concept of a market is functionally identical to (but less flexible than) an evolutionary explanation of economics.

I’m obsessed with evolution, right now. I’m sure you can tell.

The biggest reason is that evolution is arational. It is not irrational, or against reason; it simply has nothing to do with reason. It functions independently of both truth and God. It is a system which progresses – changes, at the very least – without any component of the system (or the system itself) having (i) access to the truth, (ii) direction or intent, or (iii) awareness of the system. It changes endogenously, and requires very few assumptions to work.

It’s beautiful.

Half the money I spend on advertising is wasted; the trouble is I don’t know which half.

- John Wanamaker

Recently, Andrew Eifler has been thinking about the possibility of a bubble in digital advertising.

He raises a number of good points. In summary, they are:

1. People see unlimited growth potential for digital advertising. In fact, most internet-based companies launch intended to depend exclusively on advertising to support themselves.
2. Advertising is hard to value.
3. People are buying “new media advertising products” without a clear idea of how much they are worth.
4. Therefore, there is the chance of a bubble.

I’m going to agree with Mr. Eifler’s core conclusion, but disagree with a few of the assumptions he uses to get there.

First of all, advertising is inherently difficult to measure, as Mr. Eifler points out. You can do all the advertising you want, but the only metric you really know – or care about – is units sold.

There are two problems with this metric. The first is the (classic) issue of counterfactuals. Just because you sold x number of units with advertising, it doesn’t mean you wouldn’t have sold x number of units without advertising. Furthermore, just because sales declined by e units, you don’t know if the advertising was effective, or that without it units sold would have declined by 5e units.

Second, there’s the problem between long-term and short-term sales. A great deal of advertising is “informational” – you let the right people know that your product is available at a certain price. But more and more advertising is “aspirational” – attaching your product to a particular lifestyle. It’s done, crudely, in artistic or entertaining advertising; and a bit more subtly as product placement in TV shows.

The problem is how you measure the ROI of short-term and long-term advertising. For all you know, the ROI of your long-term advertising strategy could be negative – but how the hell would you detect that?

This problem of measurement is compounded by the proliferation of new advertising channels.

Now, I’d going to disagree with Mr. Eifler and say that the digital advertising is better than traditional forms of advertising, because you have (vastly) more metrics. In fact, you can even (sometimes) track an individual all the way from initially seeing an ad to purchase!

The additional metrics makes valuing easier, though I have to give a nod to Mr. Eifler and point out that it’s possible to “worship a false idol.” That is, you could construct an elaborate and mathematically beautiful valuing equation using all those metrics… that means precisely bugger all when it comes to ROI. The temptation of having metrics is using those metrics; but metrics are not reality, they are only a reflection of reality. They are subject to both measurement error and conceptual error – where what you think you are measuring is not, in fact, what you actually are measuring (the quintessential example is that IQ tests do not measure intelligence).

However, digital advertising leads to a larger problem – which is the proliferation of advertising channels.

Let’s say you spend $1m on TV advertising. Now, we can’t make many assumptions about how the effectiveness of that advertising will change as you raise spend to $10m, but it’s reasonable to assume that the relationship between spend and effectiveness is monotonic; that is, the effectiveness of your advertising campaign will not decrease if you double your spend.

However, the relationship could be anything from linear to (more likely some kind of S-Curve:

As you increase spending early on, advertising becomes more effective, per dollar. However, at some inflection point – in the middle of the graph above – advertising begins to become less effective per dollar. The effectiveness is still increasing; but each additional dollar gets you a little less than before.

(I’d appreciate it if some people in the advertising could check me on this wildly unfounded assumption).

Now, however, let’s say you spend $10 on a bunch of different channels – TV, radio, billboards, online banner ads, online search ads, sponsored viral videos, whatever.

How do you measure the effectiveness of not only each channel, but also all channels combined? This is not a trivial question; at the least, you need to consider 2ⁿ-1 factors, where n in the number of channels.. If you have TV and radio advertising, you need to consider (1) the effect of TV, (2) the effect of radio, and (3) the effect of radio and TV together.

If you bump that up to 3 channels – let’s say TV, radio, and billboards, then you have to consider (1) the effect of TV, (2) the effect of radio, (3) the effect of billboards, (4) the effect of TV and radio together, (5) the effect of radio and billboards, (6) the effect of TV and billboards, and (7) the effect of TV, radio, and billboards together.

To make matters worse, any significant interaction at the “higher” levels will invalidate a straightforward effect of any less complex combination. So if there is a 3-way interaction, then you cannot analyze TV independently; because if you say “TV works” that is conditional on both radio and billboards; you cannot then increase TV spend independent of radio and billboard spend and except to see it work more.

If you have 6 channels, let’s say, you’re looking at 63 different factors to understand.

To introduce another wrinkle – I know, sorry – the above assumes that everyone sees all advertising channels, i.e. that everyone comes from the same statistical population.

Except they don’t. Now, it would be simple if people who saw one channel didn’t see any of the other channels; but that isn’t true. Since you are not dealing with either the same population or independent populations, you need (somehow) to divide customers into “channel segments.” That is, people who see TV and billboard but not radio; people who see online ads and TV ads but not radio and billboards – etc.

This complexifies the problem, because you now need to figure out how each channel reacts to each combination.

Additionally, the above outlines a scenario where advertising is identical, e.g. increasing spend gets you more of the same. But not all advertising campaigns are made equal – the sheer creativity in the market means that the same dollar will get you a different product depending not only on which agency you hire, but also on your company. The agency may make a campaign which would be great for a company that is very similar but not the same as you. Or one that just slightly misses your target market, reducing its effectiveness.

So: the inputs are highly variable, the outputs are difficult to measure; even if you could measure them, interpreting them would be very close to impossible, even if you could do all that it’s still a counterfactual (you don’t know how close your conclusion is to the truth, since there’s no way to test it).

Therefore, accurately valuing advertising is pretty much impossible. You just have to be satisfied with the assumption of monotonicity, and vague claims that the interaction across channels is positive.

In the end, then, I agree with Mr. Eifler. But not because new media is hard to value; I think it’s easier to measure the post hoc effectiveness, which is a pretty good proxy. The digital scene may improve matters, due to additional metrics.

However, given the difficulty of valuing advertising and the expectation of unlimited growth, you certainly have the chance for a bubble.

The company they feature is Quora, Inc. It’s founded by some very famous people – former CTO at Facebook – and was established in 2009. In March, some people invested at a pre-money valuation of $87.5 million (or: more than $10 million/employee, in a year. That’s some equity growth!). It’s particularly impressive since that was when the product was in private testing, before not even a business model but even traffic.

Quora had the bright idea to improve the Q&A space on the Internet. I mean, sure, there’s Yahoo! Answers; Metafilter and Amazon’s AskVille; Answerology, and a great deal more.

But, see, Quora has done something different:

The service allows people to pose or answer questions—working behind the scenes to route questions to the users who can best answer them.

Yes, it uses machine learning to route questions to capable users. This is a break from the current success-model, which is to treat it like a game – reward people with “points” of some kind.

And! Importantly! It’s a gap in Google’s product offerings! Google Answers was retired in 2006. Besides; Google Answers didn’t route questions to experts immediately – they had paid experts who had to trawl through the questions. It wasn’t scalable. Silly Google – no wonder they retired it!

Perhaps my sarcasm is come through a little strong.

Since 2007, Edward Chang – Director of Research, Google China – has been spearheading work in “classification and collaborative filtering.” What does that mean? Well, in 2007, Google launched Otvety (Russia) and Wenda (China) as trials for new applied of Google-researched technology in a product called “Knowledge Search.”

Edward Chang outlines this in a 2008 article outlining some of Google’s Data Management Projects:

Knowledge Search allows users to post questions and then matches experts to timely answer questions. The distinguishing feature of this product compared to competing products is that it offers online question classification, related-question recommendation, and topic-sensitive expert matching.

…. which sounds awfully similar to Quora. And it has the advantage of Google Researchers – experts in this small field of knowledge representation, question classification, and matching – working on it since 2007.

Given Quora’s lack of content, if Google simply ported the existing products over to the US, it would have a directly comparable product backed by the Google brand. And “directly comparable” is probably understating the technological advantage; you don’t put together something really sophisticated in a year, particularly if you’re busy trying to get a company together besides. Besides which, it’s not necessary; the marginal improvements in algorithms would be minuscule compared to spending their time building out the site.

So, if it wasn’t clear enough: I’m astonished at the market research which presented this as a “gap left by Google.” Yes, it certainly hasn’t been a priority; and yes, the current projects are research-orientated; and yes, it’s likely that it won’t become a priority. It’s entirely possible that Quora could become successful.

But that’s predicated on Google not doing anything with assets it already has. It’s certainly possible, and may even be rational – if Quora grows the market, then Google may decide to buy the product and (importantly) the people.

But beyond that, I’m astonished anyone would invest at a pre-money valuation of $87.5 million with this kind of situation, and absolutely no business model. And I quote:

Quora’s business model is currently unclear. Mr. D’Angelo, who became chief technology officer of Facebook in 2006 and left the company in 2008, said "if I had to guess, it would probably involve some kind of advertising at some point."

The defense, of course, is: “Find good people, let them do whatever; they’ll succeed.” And their team is, apparently, top-notch with a history of success.

I find it unconvincing.

Most notably:

Cities and neighborhoods that have received the largest influx of immigrants (including Mexican immigrants) have had — despite popular stereotypes to the contrary — the largest drops in criminal violence. (See, e.g., here and here.)

And in general, crime is declining:

Interesting stuff.

It also makes it difficult to explain why we have 25% of the world’s prison population in our borders, have a recidivism rate of 70-80%, and execute people by firing squad.