Uncertainty
Sep. 12th, 2006 05:27 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
dr_tectonicJust went to a neat seminar about project management and estimating how much time a project will take.
The interesting thing about it is that the hardest part of managing uncertainty is getting people to admit, even to themselves, that things are as likely to come in above the mean as they are under the mean. Once you've overcome that psychological barrier, thinking rationally about it is pretty easy, folks are just really loathe to acknowledge the possibility -- the likelihood, even -- that things will go wrong.
The interesting thing about it is that the hardest part of managing uncertainty is getting people to admit, even to themselves, that things are as likely to come in above the mean as they are under the mean. Once you've overcome that psychological barrier, thinking rationally about it is pretty easy, folks are just really loathe to acknowledge the possibility -- the likelihood, even -- that things will go wrong.
no subject
Date: 2006-09-13 02:36 am (UTC)Every once in a while people ask me how we can improve our estimation process.
I always reply something like "That depends... how do you want to improve it? Do you want to lower the chance that we overestimate? Lower the chance that we underestimate? Lower the average _amount_ by which we over/underestimate, never mind the chance? Increase the consistency among our estimates, never mind how accurate they are? Reduce the amount of time we spend coming up with them? More accurately identify our confidence level in a given estimate? Something else?"
They usually then say something like "We just want to improve the predictablity of our development process." To which I usually reply "OK, then I recommend we concentrate our attention on consistent PRIORITIZATION so we can identify the 2/3 of what we intend to do that we're likely to actuall do by our deadline, and treat the remaining 1/3 as unpredictable gravy."
They then run away and ask somebody else.
I'm not sure what lesson to draw from this.
no subject
Date: 2006-09-13 03:53 am (UTC):)
no subject
Date: 2006-09-13 02:24 pm (UTC)no subject
Date: 2006-09-13 03:23 am (UTC)Not that I'm cynical or anything.
no subject
Date: 2006-09-13 05:05 pm (UTC)See some of the other comments, also. Part of the idea is that if you're doing it this way, people shouldn't need to sandbag, because the safety margins have been taken care of explicitly.
I guess the question is, why are the managers trying to squeeze the extra out? Their part of the project came in early and under budget -- isn't that good? Do they want to increase the odds of having late penalties associated with their work?
no subject
Date: 2006-09-13 05:26 pm (UTC)Sadly, I have often found it is to my benefit to goof around to use up my cushion rather than turn in work early because I might need that cushion next time and my employers don't understand the uncertainties associated with scheduling. My favorite example was when the MBA running one project asked us to give her a precise estimate on when we would make what was basically a scientific breakthrough. She needed it for her Gantt chart. We tried to explain to her that we didn't know when we'd figure it out; it would depend on how the experiments went. She wouldn't budge. We made up a number; it was easier than arguing.
There are also often business pressures that are swept out of sight and treated as assertions e.g. "It must be done by Date X, and it must include all of these features or we won't be successful". In an open, honest work environment, there could be a rational discussion where the realistic estimates of what it takes to accommodate those features would be balanced with the business cases for each feature. Unfortunately, most work environments are not like that. Instead, the communication is happening through a game of Telephone such that that discussion can't take place.
no subject
Date: 2006-09-13 05:31 pm (UTC)no subject
Date: 2006-09-13 05:46 pm (UTC)Our prof said last night that a recent survey of IT projects showed that 22% were completed within the original estimate, 52% were late by an average of 100% (double the original estimate), and 26% never finished. And developers wonder why their estimates have no credibility. Admittedly, many of those estimates were poorly constructed because of management pressures, etc., but there's definitely a competitive advantage to being able to deliver on time repeatedly.
Then again, most people don't want to hear how long a software project will actually take. "It's just software! Why does it take that long?!" So even if you give them realistic estimates, they don't believe them.
I am full of cynicism and bitterness. Can you tell?
no subject
Date: 2006-09-13 05:28 pm (UTC)no subject
Date: 2006-09-24 09:48 pm (UTC)Now that they’ve reorged our company and gotten rid of all the project managers, it falls on me (as the QA guy) to give daily reports to the CFO about why the IT projects aren’t on schedule.
So I feel like maybe I should find out what the Hell you guys are talking about in this thread.
no subject
Date: 2006-09-25 02:15 am (UTC)If you've got something that follows a normal distribution (aka Gaussian distribution or bell curve), the variability of the distribution (the standard deviation) is customarily represented by the letter sigma. It's a measure of how wide the curve is.
The useful thing is that about 2/3 of the area under the curve is between (mean+sigma) and (mean-sigma). If you go out to 2 sigma on either side of the peak/center, you've got 95% of the area, and 3 sigma gets you to about 99%.
So if I do an estimate the way this guy described and it tells me that, on average, my project will take 20 weeks, with a standard deviation of 3 weeks, I know that I can come in under deadline 97.5% of the time if I commit to doing it in 26 weeks (+2 sigma). [That's assuming that the estimate is accurate, of course.]
no subject
Date: 2006-09-13 01:41 pm (UTC)Sadly, I am about the only person who understands the psychological benefits of this approach, not to mention how useful a little extra time and money can be when things go wrong.
And they do go wrong.
no subject
Date: 2006-09-13 05:08 pm (UTC)People are so unwilling to think about that. I think you're the only one there who's not just mired in denial.
no subject
Date: 2006-09-13 02:02 pm (UTC)Most people who are wise will "Scotty" their answers. Their Scotty-factor may be a percentage (15%, 50%) or it might be a whole multiple.
The unwise people give a best-faith estimate to their managers. There are usually a lot of factors the person giving the estimate is unaware of, so their number is low through no fault of their own.
Management often doesn't have well defined requirements. This means that no estimate can be good.
Management also knows that their people often Scotty their numbers. They can either add their own padding, or subtract it.
Bad management will simply pick a fixed date with no regards to any estimates.
IMO, there's plenty of blame to pass around. :-)
no subject
Date: 2006-09-13 04:44 pm (UTC)Yeah, he actually talked about how meaningful estimation depends on being able to have an adult discussion about uncertainty with your
Darth Vadermanager. If you say that you honestly think it'll take 5 months, but could shift a month either way depending on how things go, and the manager says "do it in 2", the problem is not estimation.The stuff he presented is also entirely orthogonal to padding for things like how much time you lose each day to required activity that isn't working on the project. This was all just uncertainty related to the amount of staff-days of work needed, not actual calendar time, which is kinda neat.
no subject
Date: 2006-09-14 02:37 pm (UTC)no subject
Date: 2006-09-14 03:43 pm (UTC)"Overhead" is a concept a lot of programmer folk have difficulty wrapping their minds around.
That surprises me. Just about every resource on a computer has overhead, so it's hardly a foreign concept...
no subject
Date: 2006-09-14 04:10 pm (UTC)I suggest going around and chatting with yours. Ask them how many hours of the day they think they're productive in each of their tasks that would be reflected in a project plan. I'd bet you a drink that they'll overestimate by at least an hour or two.
no subject
Date: 2006-09-14 04:33 pm (UTC)And I have no illusions about my own (in)efficiency.
Fortunately, when I am doing something productive, my output rate is usually pretty high, so it all works out in the long run...
Not a rant
Date: 2006-09-13 04:12 pm (UTC)One point: in project management land, extra time is called "padding" or "pad." In a non-disfunctional system (and really, I've worked in several, it's not actually impossible) you gather the actual time estimates then add in a standard amount of pad to cover what you don't know.
However, on a different point, you also cannot schedule something if you don't know what people want you to build. However, you CAN schedule something if the people who are defining the requirements agree to consult you: "Can you do X and Y in the time you have left?" If you can't get it done, that feature is automatically scaled back or dropped. Alternately, everyone agrees the date must be moved.
Again, I have worked in places (and do currently) where this really happens--people are really interested in doing their best to overcome these kinds of challenges.
But here's a question, Dr. T: there is one theory that says that people generally over- or under-estimate how much time something is going to take by a fairly consistent factor. Assuming you can identify a few variables (Sue always thinks db work takes longer than it does, Bob always thinks he can just whip out the help files and forgets they have to be approved by the marketing department before he's done with them) you can get a pretty good idea of how to adjust the estimates to be more (but not perfectly) realistic. The key, however, is never to tell the people doing the estimating how you adjust their numbers, or they'll try to compensate which throws everything off.
It seems a bit underhanded, but in a lot of ways this approach seems like it makes a lot of sense. Did the seminar touch on this kind of approach and say whether it's a good or bad idea?
Re: Not a rant
Date: 2006-09-13 04:56 pm (UTC)He talked about it as being primarily a way of being honest with yourself. Instead of asking just how long will it take, you ask how long will it take on average, how quickly could it be done if everything goes well, and how long might it take in the worst-case -- and then you combine those. If you ask Bob those questions, you'll get a different, and probably much more accurate, number, even given his tendency to underestimate (which you could then account for in the way you describe).
Re: Not a rant
Date: 2006-09-13 05:17 pm (UTC)This was dramatically illustrated to me in my first couple months working as a consultant, where we were asked to add a single feature to an already working test system. I wanted to do it right then and there, and my boss said "We'll go to lunch and get you a good estimate". He asked me at lunch my estimate, and I said 4 hours. His estimate was more like 16 hours after including all of those elements I listed above. Guess who was closer to right?
And, yeah, then you have to take into account meetings, coffee breaks, lunch, picking kids up after school, vacation, sick days, etc. It adds up - I think I've read that 20 hours of actual development in a week is doing really well, and that fits with my own experience.
no subject
Date: 2006-09-13 10:08 pm (UTC)Perhaps now is the moment to do so, despite this post.