Your plan is probably wrong

The typical planning process, using a step-by-step approach, leads to unrealistic plans

Best-case scenarios with a fudge factor

Typically, when we plan a task, we start to internalize all the steps required and how long the steps will take. Again typically, we assume almost a best case scenario – all the resources will be available, I’ll be mentally sharp that day, the equipment will work, I won’t get distracted on other things, etc. With this rosy picture of how everything is going to work out well, we plan our timelines and costs. Sometimes, we add a fudge factor to account for one or two steps going wrong. If we tend to be pessimistic, we add a few fudge factors. The end result is a forecast of how the project should work, given a few mishaps.  Yet  in actuality, more things will go wrong with a project than will go right. This is why we often cite Murphy’s Law. Unfortunately, we usually think of Murphy in hindsight, not during planning.

Even when presented with evidence of how past examples took longer, we find exceptions in the history that shouldn’t apply to the future. For instance, it should only take me 3-4 hours to clean out the garage this weekend even though it took me two days the last time I did it. The reasons it took me longer last time was because I found a box of albums from my college days and reminisced over those for a while, and then I found a box of the kids’ old toys, and, again, I reminisced. After that, I found some of the neighbor’s tools and returned those to him, and we started chatting, and he ended up showing me his new home theatre, and I stayed and watched a movie. I’m sure nothing like that will happen this weekend when I clean out the garage.

The reason why so many home improvement do-it-yourself projects get abandoned half-way through is because we end up encountering so many unexpected difficulties. In the do-it-yourself guide, replacing the toilet valve looks so easy, but when we  try it, we find the pipes are rotted and can't turn off the water. Then, we can't remove the old hardware because it is corroded. When we finally remove the old, rotted hardware (having replacef the pipe and fittings in the process) we find that we actually have a non-standard sized toilet, and our new parts don't fit correctly. At this point, we abandon the project and tell everyone in the house not to use the toilet.

This is actually what happens at work, too. We plan a software upgrade and find that our old operating system doesn't support the new version so we have to upgrade that first or we develop a new streamlined process and discover we have to rewrite all of SOPs and then train everyone on the new SOPs or we buy a new piece of lab equipment and realize that we have to test it rigorously before we can integrate it with our systems. Nothing is ever as easy as we first planned.

 Internal focus (where everyone is above average)

In addition to adopting the best case scenario as our baseline, we also tend to think that our project or situation is unique. Somehow we possess the knowledge or the skills or the luck that everyone else lacks.  Garrison Keillor nailed it when he described Lake Wobegon as a place where all the children are above average. In studies of how accurately people can judge themselves, most people rate themselves as having above average intelligence and above average athletic ability. This is why, when we plan, we think "It may take you three weeks to accomplish that task, but I can do it two."

 In study after study, psychologists have documented the tendency of people to underestimate how long it will take to complete a task, even given a history of their performance in the past. Thus, telling people to use better planning techniques and to err on the conservative side has little effect. People will still create an overly-optimistic forecast despite history to the contrary. This common phenomenon is called planning fallacy. Planning fallacy is a fancy term for "the tendency to hold a confident belief that one’s own project will proceed as planned, even while knowing that the vast majority of similar projects have run late.” (Buehler, R., Griffin, D. and Ross, M. 1994. Exploring the "planning fallacy": Why people underestimate their task completion times. Journal of Personality and Social Psychology, 67: 366-381. ).

Psychological studies also show that people do a much better job predicting how long it will take others to perform a task than predicting their own performance.  The difference in the thinking is that when we judge others’ performance, we use “outside” information like how long similar tasks take and how they have performed in the past. When we judge our own performance, we use internal narratives that discount our histories with exceptional situations (otherwise known as making excuses), build an optimistic step-by step picture of how we think things “should” go. Finally,  we are felled by the “better than average” bias into thinking that we can do a much better job than those other people in their projects, kind of like thinking that, “It won’t happen to me.”

The very simple way of eliminating the planning fallacy is to base predictions on the knowledge of how similar projects fared and use that as the starting point rather than the internal step-by-step of how it “should’ proceed. Once you have that scenario, plan for pretty much everything to go wrong, a la Murphy’s Law. After you get that estimate, multiple it by 3 and now you have a workable plan.