« Gulf Dead Zones (Before BP) | Main | Dumb Things that Seem Smart »

Due Dilligence v Data Sufficiency

Studies show that you feel more confident about a decision as the amount of information you have grows.  Unfortunately, the same studies show that increasing information does improves decision making at a diminishing rate and at some point, usually around seven discrete pieces of decision criteria information, the efficacy of decision making actually declines.  But confidence continues to rise. It is usually a bad thing when confidence overtakes capability.

I am facing this problem all day, every day, on my promotion panels. I take this responsibility very seriously and I feel empathy with everyone I am judging.  Aways I am looking for the additional piece of information that will make my decision more certain and my certainty grows with each additional fact I find.  But then I recall the science on human decision making.  Perhaps my certainty is unjustified and undesirable.  I feel better about what I am doing but I am not doing any better.

Seven is the magic number or maybe the logical limit.  It is no coincidence that many thing in literature and myth come in sevens.  You have the seven deadly sins, seven wonders of the world, seven samurai, seven habits of highly effective people, seven voyages of Sinbad etc.  It is also why telephone numbers originally had a maximum of seven digits, of course broken up into smaller subsets.  Our minds simply aren’t well designed to keep lots of information both available and sorted.  We are not the logical decision makers portrayed in the movies and maybe in our own minds.

So we make lots of our decisions using heuristics – rules of thumb roughly derived from experience and previous successful decisions.  These work fairly well if the current situation is well understood and analogous to past ones and if we are aware that we are indeed using these rules, so that we can make appropriate allowance and adjustments.  I think a lot of this is happening in our judgments.   I also believe that it is working reasonably well because of the diverse and relevant experience represented on our board.  Of course, I have to keep reminding myself about that problem of growing confidence and declining effectiveness.

Sometimes you just know all you can. It may not seem enough, but it is time to decide. All decisions are made under conditions of some uncertainty. Otherwise there is no need to make a judgement.

Our scope for serious errors is not great. Almost all the people we are considering are well qualified.  Beyond that, we reach fair convergence among those near the top and the bottom of the distribution. This tends to happen BEFORE we talk it through, so it is not based on overt groupthink. Since the numbers of promoted and low ranked are relatively small percentages of the total, I think we can be confident that most of those “belong” or “deserve” to be where they ended up.  I worry a little about those right near the cusps.  But I don’t see any way to increase effectiveness, although it is pretty easy to increase confidence.

Hosting by Yahoo!


Bearing in mind that a person's performance can get better or worse, often surprisingly, and that they are all well qualified, why not just pick them at random?


Sometimes I wonder whether or not randomness would work.

It seems to me that the very top and the very bottom are easy to find. Since we don't promote very many, I am fairly sure that almost all those who make it deserve to make it. I worry more about those that don't.

Of course, there is also the question re whether we are selecting for the right skill sets. But that is a decision made above my pay grade.

I am impressed by how "fair" the process has been, if you define fair as giving everyone a similar chance and not interfering with the process. There is absolutely no political or organizational pressure on us. Although I suppose that we have been in the FS so long that we all are tame and predictable so nobody has to bother us.

We have a similar problem in Britain with candidates for Oxford and Cambridge universities. Many times more highly qualified candidates apply than there are places available. Inevitably most fail to get in. There are regular accusations of class bias etc, which I think are unfounded.

Attempts to be fair lead to a formalisation of the process, so that those who are good at jumping through the officially approved hoops do well. In the case of the universities, the primary requirement is very high marks in the standard school-leaving exams ("A" Levels). This excludes any non-conformists.

Tame and predictable people select tame and predictable people. Thus society becomes mandarinised.

Just some thoughts - I don't want to cause you any doubts and worries. ;-)

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)