Word Clouds

I wrote a short tutorial explaining how to make a word cloud suitable for including on business cards. It can be downloaded here:

http://tinyurl.com/WordCloudTutorial

Common Sense vs. the Monty Hall Problem

This post isn't really about the the Monty Hall Problem, although it will be discussed at length. It's really about three characteristics of common sense.

Common sense:
    ... is fast.
    ... is usually right, but sometimes wrong.
    ... gives us overconfidence in it's answer.

I think evolution can provide an explanation about why common sense has these characteristics. Suppose our ancestor was in a life-or-death situation and had to chose the right course of action to survive - let's say a lion spotted our ancestor from across the clearing. The ancestor can climb a nearby tree, or run back to where his hunter clan is gathered in a cave, and they can fend off the lion with spears.

There's not a lot of time to decide, so common sense needs to be fast. There's strong evolutionary selection pressure favoring those who decide quickly.

There's also strong selection pressure to make the right choice. But even if the choice is wrong, it's probably not as dangerous as doing nothing while thinking about it.

Finally, our ancestor had to have enough confidence to act decisively before becoming lion food, so there's also strong selection pressure to have confidence in ones common sense.

And now comes the Monty Hall problem.

The Monty Hall Problem comes from the game show Let's Make a Deal which is hosted by Monty Hall - hence the name of the problem. In the game, you are presented with 3 doors. Behind one door there's a good prize - let's say a car. Behind the other 2 doors there's a booby prize - let's say a goat.

You are asked to guess which door holds the car, so you choose a door at random. Then, before revealing what's behind the door you chose, Monty Hall will open one of the doors you didn't choose revealing a goat. He always opens a door which has a goat behind it - never a car.

OK, so now there are 2 doors left, and one of them has a car behind it. Monty will now ask if you want to keep the door you initially chose (stay the course) or switch doors (flip-flop). The problem is: what should you do now - stay the course or flip-flop? Or does it even matter?

This is where common sense usually tells us it doesn't matter, as there are 2 doors and the car could be behind either door giving a 50/50 probability either way. But common sense is wrong here.

At this point it's common to have overconfidence in the common sense answer. This is just a natural feature of common sense.

We can detail every possible case for when you need to decide to stay the course or flip-flop as shown in this table. The numbers in red indicate the probability of passing through a given cell in the table.

The Monty Hall Problem

Case #	If car is behind door #	And you chose door #	Monty Hall will reveal goat behind	You should
1	1    (1/3)	1    (1/9)	Door 2 or 3	Stay the course
2		2    (1/9)	Door 3	Flip-flop
3		3    (1/9)	Door 2	Flip-flop
4	2    (1/3)	1    (1/9)	Door 3	Flip-flop
5		2    (1/9)	Door 1 or 3	Stay the course
6		3    (1/9)	Door 1	Flip-flop
7	3    (1/3)	1    (1/9)	Door 2	Flip-flop
8		2    (1/9)	Door 1	Flip-flop
9		3    (1/9)	Door 1 or 2	Stay the course

Of the 9 possible cases, in 3 cases you would have chosen the door with the car, so you should stay the course. In the other 6 cases where you chose a door with a goat, flip-flopping would get you the car. Taking all 9 cases into account, flip-flopping will win the car twice as often as staying the course.

So when Monty Hall opened the door with a goat behind it, why did the probability change to 33/67 instead of 50/50? It has to do with the fact that when Monty Hall opened the door, he was only going to open a door with a goat behind it - never a car.

But what if the game had gone as follows:

You are asked to guess which door holds the car, so you choose a door at random. Then, before revealing what's behind the door you chose, Monty said you can keep what's behind the chosen door, or you can have the best of what's behind both doors you didn't choose. In this scenario it's obvious that flip-flopping will win the car twice as often as staying the course.

Alternate Scenario

Case #	If car is behind door #	And you chose door #	If you flip-flop, Monty Hall will reveal what's behind	You should
1	1    (1/3)	1    (1/9)	Doors 2 and 3	Stay the course
2		2    (1/9)	Doors 1 and 3	Flip-flop
3		3    (1/9)	Doors 1 and 2	Flip-flop
4	2    (1/3)	1    (1/9)	Doors 2 and 3	Flip-flop
5		2    (1/9)	Doors 1 and 3	Stay the course
6		3    (1/9)	Doors 1 and 2	Flip-flop
7	3    (1/3)	1    (1/9)	Doors 2 and 3	Flip-flop
8		2    (1/9)	Doors 1 and 3	Flip-flop
9		3    (1/9)	Doors 1 and 2	Stay the course

As it turns out, both scenarios are statistically the same. In the original scenario, he showed the goat you wouldn't want anyway from among the 2 doors you didn't select. In the second scenario, he let you discover the goat you wouldn't want anyway when you opened the 2 doors you didn't originally select. From a practical point of view, it's the same game, and flip-flopping wins 2/3 of the time.

The thing to learn from all this is that we should use common sense when we need a quick answer. After all, that's what common sense is for, and it's usually right even though it can be fooled. Where we should not rely on common sense is when it's important and there's plenty of time to do the analysis. And we should always be wary of the confident feeling that comes with common sense.