Estimation: why can’t we get it?

In software development shops across the world there is absolutely nothing more frightening to software developers than estimating a task. The fear stems from the unknown. The phrase “I don’t know what I don’t know” comes to mind.

The pressure of giving an estimate for a task in terms of amount of effort can be very heavy. Somehow telling someone how long it will take to get this task “done” has turned into a self defense mechanism. So-much-so that sometimes engineers “pad” their estimate to almost twice what they believe it will actually take.

And then, the project managers get ahold of the estimate and they pad another 50%. This brings the estimate to 3 times what the engineer actually thought the task was going to take. And you know what happens? It takes that long or longer, but rarely does it take shorter.

Why? Why do we always end up taking longer than we originally thought? In my experience there are a few reasons for this phenomena. Each of which has played out in development shops that I have been a part of.

The first is the self-fulfilling prophesy syndrome. This is when a developer fills the estimation time because he has the time to fill.  In this scenario, if you give an engineer 8 hours to do the work, he will take 8 hours to do the work. In my estimation this is the worst kind of situation a development shop can be in. Because this is a good indication that your engineers are bored and do not have a vested interest in the shipping of the product.

A second reason estimations are off the mark is because engineers don’t learn from what they have previously estimated. Even green engineers right out of college have some experience estimating. They do it every day of there live in college. How long will it take to get this homework done so I can head out with the guys and play Gears of War? When an engineer is able to take into consideration previous work/estimate relationships and contrast with the complexity of his current task, that engineer is already more accurate.

Another reason is because the task is to great to estimate accurately.  This is the one thing I see more often than anything else.  An engineer has not taken the time to break down the task into manageable chunks. I like to pose this question to my team: Which estimate is going to be more accurate? 1. How long will it take to drive from downtown Raleigh to downtown Durham given moderate traffic conditions? or 2. How long will it take to drive from downtown Raleigh to Times Square in New York City? Of course we all know the answer. Because when we estimate small chunks we are more accurate.

There are more reasons than these but these make up a great percent assuming it is the engineer giving the estimate.  If it is not then there is a bigger issue that needs to be addressed in the organization.  So when an engineer thinks a task is going to take 4 hours he should estimate 4 hours and not pad anything.  He should then take into account how long it did take and what happened during that time. He should use this knowledge the next time he needs to give an estimate. He should also learn to be breaking down his tasks into realistic estimate-able chunks. Some say if it takes longer than a day then it is more than one task. I am not going to be that stringent but I think that is a good target.

Estimating is a skill that gets better with practice.  In the Agile development world we give estimates every iteration and every day. It is the only way to get better.  Keep at it and don’t get discouraged.

Till next time…

Technorati Tags: Estimation,Agile