The myth of Project Management

I first discovered the term “consensual hallucination” when reading William Gibson’s seminal cyberpunk novel “Neuromancer”. He used the term to describe his concept of cyberspace – a place that didn’t actually exist but it was a representation of a massive computer network that people could conceptualise well enough to deal with. For ease of understanding, everyone treated an abstract concept as a physical reality – they consented to believe in the hallucination because it was easier to understand that way.

Life is full of consensual hallucinations. If you live in a democracy, you tend to believe you have a say in what happens even though the whole political system is completely under the control of vested interests. Voting is a sideshow but we tend to indulge in the consensual hallucination that it means something because life is easier to bear that way. In fact, pretty much any political or religious belief system is a consensual hallucination.

Which is not to say that they are all by definition untrue – simply that they are abstract systems that people tends to treat as concrete reality despite the complete lack of empirical evidence to support this consensual hallucination.

When you work in IT, you deal with the consensual hallucination of Project Management. There is an almost universal belief that it is possible to predict ahead of time how long a project will take, how much it will cost and what will happen along the way. In the broadest sense, this is possible. If you have enough experience you can come up with ballpark figures; last time we did something similar it took this long and cost this much.
But some people believe Project Management should tell you these things down to the day and the dollar. A project plan should tell you every task that needs to be completed. A project plan should be flawless and leave nothing to chance. And a project plan should be completed before ANY work is done on the project.

Despite the fact this is clearly insanity, it is a terrifyingly common mindset in management ranks. Project planning and goals are obviously important at some level (otherwise how the hell would you know what you are doing?) but how did we move from “let’s have a clearly defined set of project goals and a strategy for how we’ll get there,” to “this is 100% accurate, carved in stone and will never change”?

I think there are a number of reasons the myth of Project Management has been elevated to the level of holy scripture:

  • A whole industry of consultants who base their existence on the lie that they can provide definitive solutions to Project Management
  • Several rainforests worth of books purporting to offer the definitive methodology for flawless Project Management
  • Nobody likes to look stupid. If you’re a professional and someone puts you on the spot to answer “how long will this take?” you like to have an answer
  • When you do work that is measurable in retrospect (“we wrote this many lines of code and it took this long”) it’s easy to fall into the trap of thinking you should be able to project forward accurately (“it should be this many lines of code which should take us this long to do.”)
  • Very few businesses are keen to hand over an open chequebook – when you buy stationery, you know what you’re getting and how much it will cost. Why is it so hard to get the same thing with software?

So how do we escape the consensual hallucination that there is a way to do Project Management that is absolutely foolproof and provides definitive answers?  Well, the first step is to kill all the consultants.  Okay, maybe that’s a bit harsh.  Let’s limit ourselves to killing the consultants who act as though they have some mystical powers that enable them to succeed where all other have failed.  Maybe even that’s going a little bit too far.  Surely there’s a solution that doesn’t involve the risk of incurring jail time?

There is no silver bullet (although there’s quite a good essay entitled “No Silver Bullet“) that will solve this issue but there are things that can be done to improve the situation.  How about we all sit down to a big three-course serving of reality together?  If you’re on the development side, one of the best things you can do is to have the courage to say “I don’t know,” when you don’t, in fact, know the answer to a question.  Sure, the geek mystique might take a bit of a hit when you admit you’re not infallible but it’s worth it in the long term.  If you’re on the business side it means LISTENING when your IT people tell you that a definitive answer isn’t possible.  The obsession some people have with getting an answer, any answer, even when they know the answer is totally inaccurate is nothing short of stupidity.  Just don’t do it.

It’s perfectly reasonable (and I would even suggest that it’s necessary) to come up with some sort of estimate in a project plan.  So long as everyone agrees that’s what it is: an estimate.  And, more importantly, the estimate will need to be revisited at key milestones to see if anything has been uncovered that will affect previous estimates.  For this to work, everyone has to be open and everyone has to listen and be responsive.  Or we can keep flailing about saying it has to be this way because the sacred Project Plan says it’s this way.

To illustrate the ups and downs of Project Management, in another post I’ll be documenting the project I’m finishing up on right now.  I’ll be using my current project as an example because it’s one of the best managed projects I’ve ever worked on (I’m not the Project Manager so don’t worry – I’m not big-noting myself).  It’s a good illustration because it’s quite a big project and most of the Project Management was done really well but there were still some problems.  And at the end of the day, it’s the way everyone involved handles the unexpected that drives whether a project will succeed or fail.

About these ads

52 Comments

Filed under Work

52 responses to “The myth of Project Management

  1. The issues you have brought out are covered in spirit of popular methodologies like
    PRINCE 2 and PMBOK.
    If you read the concepts of Integrated Project Management, you may realise that the Project deadlines are never sacrosanct, inflexible or frozen but are ever evolving in response to dynamic project environment and stakeholder interest. What it emphasises is that all changes in schedule must be thoroughly analysed for impact , their work arounds considered and concerned stakeholder be apprised. It inculcates the fighting spirit to defend your plans and also to accept as inevitable some factors which might have been reasonabe.
    It never tells to apply all processes by book.. PM as envisaged globally is a collection og globally established best practices which are logical and rational. They are at best tools just like your PC.
    A

  2. I totally agree with Arun.

    Every project needs to be flexible. The attack on the World Trade Center is exactly the kind of risk that can never really be allowed for during the planning stage, but they can happen.

    An estimate that isn’t 100% accurate is just another risk to be managed.

  3. I agree with Arun on this completely also

  4. Pingback: Project management « The notes of the tame cockatoo

  5. Hmmmm, so project management works except when it doesn’t? ;) That’s a pretty handy response.. follow this process except when this process is wrong…

    Actually, I agree with Arun too, the most important quality in project management is adaptability. And regarding the WTC, impact by an airplane was a risk that was assessed in design, it’s actually a very obvious risk. The building were designed to withstand such an impact and they actually did. The problem was the almost full fuel tanks burning hot enough to melt the steel supports.

  6. In my Software Engineering class at Western Washington University, the students were taught two methods of estimating how long a project would take.

    Optimistic method: Take your initial estimate, and double it.

    Pessimistic method: Take your initial estimate, double it, and move it up to the next highest unit of time.

    For example, if I guesstimate that a project will take 2 weeks, a more realistic (optimistic!) estimate would be 4 weeks, a pessimistic estimate would bee 4 months. (double 2 weeks to 4 weeks, then move weeks up to months).

    And yes, constant realignment to reality, needs to be performed. Engineers tend to say “oh well this one thing took me twice as long as expected but I’ll catch up and get the next item done twice as fast.” Bad idea. A better estimate would be “Wow, I had better plan on the next item taking more time as well.”

  7. I have worked in IT at the LAN level and also on the carrier (WAN) side for 15 years.Both designing and managing ISPs and building global networks for Lucent Technologies (anyone remember LU ?)

    The Myth of Project Management was superb ! The heretical nature of project management seems to be a never ending source of entertainment, along with the inherent joys of IT.

    Rock On AA :-)

  8. Most of it is common sense

  9. Salamaat,
    I am with you as always Mr. Angry, the project I am on is a mess when it comes to PM and as a last minute grasp at straws they decided on a *firm* deadline for Design phase…only to have BA’s scampering around trying to get sign offs on time etc.

    You know the end deal, we got the sign offs but a lot of short cuts had to be taken and now people are going backwards to fix the holes.

    it’s quite retarded.

  10. Devlin: nicely put! Would you be surprised if I said I liked the pessimistic approach?

    Clark: thanks :) Nice to see another experienced voice agreeing!

    lux: exactly, common sense! Why is it in such short supply in workplaces?

    Maliha: Salamaat. Yeah, sign-offs are a hott. They usually take more time than the actual work. That’s going to be the feature of my follow up to this post – teh one thing that went wrong on this project was sign-offs.

  11. Great post. I always liken it to, especially in Academia, no one wants to tell the emperor that he has no clothes. I’ve been involved in numerous projects that were successful only because we didn’t advertise them to the consensus by committee approach in Colleges and Universities.

  12. :-D IT and Project Management are like oil and water. You can try to get the two to mix but they never really do because one of them is slimy and coagulates without bonding with the other. Take your pick on which one is which.

  13. marrngtn: I agree with the emporer’s new clothes analogy. I personally don’t see the problem with saying “we don’t know for sure how long this will take but we have a plan and we’ll revisit it at key milestones to see if anything has changed.”

    jaredude: hahahaahaha, nice call.

  14. Another consensual halucination: A jury of one’s peers.

  15. But isn’t that the key to being a good PM? Working with a plan, and an estimate, and the ability (by being able to physically say, and to have the environment/culture to express) to say that things will be revised/revisited as necessary?

    I recently took a class on the PMBOK, as I’m (sort of) studing for my PMP. The instructor expressed that the tools/techniques are sort of a best practice for the field of project management, and it is up to each individual PM to determine what every project requires.

  16. “The more you plan the luckier yout get.”

    @Kate: agree on the cultural aspect. Working with koreans I see this every day. Despite better knowledge the project milestones are scarved in stone because promises which were never realistic had been made.

  17. wolffood: I’m glad someone responded to that asopect of the post!

    kate: I essentiaklly agree – in the real world you will find far too many people who treat project management as an absolute when in reality it is constantly shifting.

    eckboss: isn’t that the worst feeling ever? People refusing to see the insanity of sticking to a deadline simply because someone said it would be so?

  18. Pingback: the myth of project management « asocial studies

  19. @Luxgood: While it is common sense, sadly, very few people have much if any common sense.

    I always lean to the heavy side on estimates because I know that my estimates are going to be cut anyway. Usually I get what I wanted without them knowing.

  20. As a Project Manager, in my experience, the biggest issue is not so much about defining a deadline or a timeline, but getting the client to accept and appreciate it.
    As long as you can keep them believing in good faith that you’re not out to scam them like some sort of mechanic telling them “Well, I fixed the tranny, but now the alternator’s shot. . .” and milking the clock, so to speak, then things should go right.
    In my line of work, you get clients that can understand that things aren’t always so cut and dry, but you also have clients that expect a clearly-defined deadline and for the project to not go beyond the deadline.
    I’m not one for cutting corners, by any means, just to get what is ultimately the illusion of a complete project, but sometimes the client’s attitude decides whether or not that is neccessary – especially if they don’t sign you on for any further maintenance ;)

  21. I currently consult for an international financial firm and deal with manufacturing. There are deadline driven projects and all others.
    Anything that is not driven by a deadline will unltimately use all the time available. Any project that has a deadline will ultimately fit all the requirments into the time allowed. And then there is failure. Failure is only assigned on deadline driven projects. Of course then the project either gets canceled, or the deadline extended. Or, you get to do it all over again.
    Measure twice, cut once, as an old hillbilly carpenter once told me. In today’s world, the Japanese and Koreans typically do it the right way, like plan do act, plan do act. Here in the US we like to drag everything out and then reinvent the wheel at the eleventh hour.
    In truth Project Management is an oxymoron. The project manages you. If you can manage yourself during the project you can succeed. Expect the unexpected, plan for the worst and hope for the best. Keep good records and review review review.

  22. There are two reasons project management is important:

    1) To show incremental improvement from day-to-day or week-to-week.
    2) To have answers when your boss ask questions.

    Project management is enhanced with proper change management. If a time line slips, there needs to be a documented reason. If possible, that document should have a customer signature on it. A timeline is only late if the customer believes it is late.
    Perception is reality!

  23. Interstellar: the weird thing is, I’ve heard so often from management that they know IT is inflating figures so they always force a cutback which is ridiculous. It’s saying “we’re all lying so let’s accept that.” How about we tell each other the truth?

    mhargrave: you’re right, in a perfect world, the estimates are a conversation. There’s some back and forth where you negotiate an acceptable timeline.

    9t84well: that’s a pretty good approach

    Doc: now that’s an approach I can live with!

  24. Pingback: Mike-O-Matic » 6 Signs of Good Software Project Managers

  25. Pingback: The myth of Project Management « The Business Intelligence Developer

  26. Many good points here in both the blog entry and the comments. I have a long and bad relationship with project management. Funnily enough, I get on well with Project Managers, just not their chosen profession. As a technical lead on a number of projects, I have grown used to having my estimates halved or thirded before they get passed on to upper management. Of course, they don’t like it when the project comes in closer to my estimate than their doctored version of it and I say “I told you so”!

    I think another problem with project management and estimates is the concept of fixed salary, no overtime and poor job markets. You see, just about every project starts off with only two things: a fancy name and a deadline. Then even though the project might take 2000 hours for a 1000 hour planned duration, the programmers will usually hit the deadline by working all hours and about burning themselves out. Thus, the feedback that management takes from it is that programmers lie when they say the need 200 hours and they’ll get it done with whatever crazy deadline that management wants to set.

    But I could just be cynical.

  27. Simon, you don’t sound cynical to me, just experienced and realistic!

  28. Geekmouth

    There are 3 key factors for any project – time, money, resources (people). The deeper myth of project management (held by clients and upper management types) is that all three of these can be constrained at the same time. That can never work. If you have a set in stone deadline, you can’t get by with inadequate amounts of money or personnel. A good project manager will fight that fight at the beginning of the project – or a good company will have realized this in the first place and have a project management process that is realistic.

    Simon, you aren’t being cynical. Unfortunately, you are dealing with PMs whop are putting their success above the success of the project and screwing you and your people so they look good. Unless a technical lead is known to use the “Scotty” approach (multiplying the real time estimate by 2 or 3 to make hitting their goals easy) your estimate should never be cut. In my experience, Technical Leads are aften optimistic about how things will go, so if anything I increase their duration estimate in my project plans. IMHO, the project manager has a duty to not overwork the programmers and technical resources.

    Overall, a good post.

    –Geekmouth

  29. I agree with your 3 factors, I usually tell managers they need at least two of them in abundance. And if you reduce one, the amount needed of the other two goes up. Ideally you have all three in sufficient quantities.

  30. Pingback: links for 2006-11-14 « Steve Miller’s Blog

  31. Neil Watson

    Has anyone here read Critical Chain? It’s a different approach to project planning based on the Theory of Constraints. The idea is that project plans go wrong due to Murphy’s Law (the random nature and impact of problems) and the Student’s principle (always do everything at the last possible time). In other words, typically, if you are asked to estimate the length of a task, you make a good guess and then add on a percentage to allow for problems. However, because you know you made the allowance, you allow yourself to use that time for other urgent tasks, so if anything does go wrong, as it usually does, you don’t actually have any slack time left. Leading to missing deadlines.

    The CC theory is that instead of building float into every task, the float should be built into the project itself and only the best guesses used for the plan. Missing deadlines is not a big issue unless it impacts the critical constraint (and there is only 1 in the whole plan).

    It’s a good read and there are back up books that go through the theory in more detail. See the Goldratt site, http://www.goldratt.com, for more info.

  32. Thanks for the info, Neil.

  33. Pingback: Labnotes » Rounded Corners - 62

  34. Wow! You packed a lot into this one. As a “certified PMI Project Management Professional” there is only one thing that disturbed me about the article: it hits a little too close to home. Project managers get beat up by “real” management and have no direct control over the project team, so it takes a strong, self confident and intelligent PM to overcome the tendencies you give. The keys are:
    * be honest up front with the estimates
    * have the team members re-estimate weekly the effort needed to complete the tasks they are working on
    * actually track the project with the schedule (i.e. not just use it as a checklist) to see critical path and make adjustments
    * clearly communicate the status and realistic forecast on a regular basis.

    Not doing these things would be like a quarterback running the ball up the middle every time for a loss without adjusting the game plan.

    Under bidding to get the job, hiding reality in hopes of catching up later and waiting to the end to say it can’t be done are signs that the PM is in the wrong profession.

  35. Yep, quarterback running up the middle – sounds like what I’ve seen management force PM’s to do time after time. Then they blame the quarterback for getting wiped out.

  36. I believe Mr Angry is 100% right…. The principals of Prince 2 and other crap like it exists but management doesnt give a shit… They want the numbers to stack up and they need to say at a board level that the figures we were given were wrong but we applied the very best in the industry so if they cannot do it, what chance have we got… Then they finish their champers and collect their fat cheques and shares and leave the table… For others to keep the wheels running..

  37. david: yeah, nobody believes it but they make us do it anyway.

  38. Although much of what you say is currently true of IT projects in practice, as a professional developer & analyst who moved into business analysis & project management in the late 90′s, I believe that most of the issues relating to this topic stem from the PM, planning, budgeting, and risk management abilities of the Project Manager and his/her ability to properly engage the stakeholders – and of course the acceptance that planning takes time, effort and money (which most businesses do not properly allow for during the planning stage).

    I am not someone who attempts to manage down to the day what resources will be doing what, however, it is possible to use established averages to experience to determine costs and schedules within a small margin of error.

    IT has a very poor name for delivering to budget and timeframe for a variety of well-understood reasons. However, when compared to other industries involved in one-off builds (by definition a project manages a one-off “build” of something), IT is almost unique in its poor results in planning.

    Take for example civil works such as building bridges, major freeways, 80 story buildings, building an oil platform, etc. How is it that IT can’t deliver a $1M project to schedule and budget when working with one vendor and 1/2 a dozen stakeholders, when public works efforts deal in the 100s of millions or billions with sometimes hundreds of vendors and dozens of stakeholders – and deliver to budget (with bonuses for delivery ahead of schedule), and sometimes even surpassing client expectations?

    It’s a rhetorical question, but the answer lies in the planning and understanding of the nature of what you are building. Many IT projects rely on estimates from “coal-face” IT specialists who are not trained (and refuse training) in how to properly estimate and prioritorise their own work and the work of their teams.

    Generally, IT Project Managers need to ensure that they properly understand the nature of the project (and should seek help from project managers with subject matter expertise if they are not competent in IT projects), they need to understand the confidence / error with which “coal-face” professionals provide estimates, and have the ability to develop plans in the specialist IT area then push through the budgets and plans required to deliver to schedule.

  39. I use a variation of Geekmouth’s ‘three factors’, although we both cover the same ground.

    I think in terms of ‘technical, schedule, and cost’, where ‘cost’ is shorthand for all resources.

    This formulation parallels the classic offer that a good project manager can make to the sponsor: “Fast, Good, and Cheap – pick any two!”.

  40. Talon Jensen

    My variation of the “three factors” is “time, cost and features”. I use these because they are usually what the client or user is most interested in controlling and can control.

    Time and cost are, of course, related as more time is generally increased cost (unless you drastically cut resources).

    But features is a very important factor and something the customer/user truly controls. Features is generally inversely proportional to cost and time. When faced with longer cost/time than desired the customer/user will often decide some features are not as essential as originally thought.

    In this way I try and have the customer control the software project. Frequent meetings and discussing the current state of these three factors generally results in more customer/user satisfaction. Which generally results in more satisfaction for me. :-)

    Talon

  41. I’m with you Talon, without active customer involvement, things have a way of going horribly wrong.

  42. These “Methodologies” are meant to be customized. Every organization has to internalize what works best for itself. You cannot be lazy and expect to buy an OOTB PM solution.

    I found your blog because I just wrote about this. Its interesting, but has aspects of truth and misguided PMs.

    - Josh
    MIT Technical, Boston

  43. Tikitime

    @David Maxwell:

    A silly comparison – real world, physical structures that have been build before can certainly be planned down to the hour.

    Writing software is just that: writing. Many things can go wrong along the way, starting with end users not disclosing their actual process to create the report, data or whatever.. then you find out LATER after giving a pessimistic estimate of your coding time.

    apples vs Oranges Mr. maxwell.

  44. Hey Mr Tikitime – sorry, but you just don’t get it. Every business solution under the sun has been written – just not by you. After over 40 years of business software development, there is nothing new.

    The fact that languages change and new, inexperienced people enter the industry and forcing loss of knowledge, gets interpreted by “coal-face” coders as that what they are doing is original.

    To take an example from left-field, if you were writing a paper on economic theory, would you research the web and paraphrase someone else’s work, or would you set about creating a whole “new” economic theory that matched what you thought others expected? Only in software do we assume that we can do it better than anyone else before, and thereby ignore most of what has gone before.

    I have been writing software since 1987 (prior if you include university and school). The problem is that software developers see the world as you do – it’s all about code.

    However, after decades of analysis and exhaustive studies around the world, code “writing” is shown to vary as a percentage of total project time in development projects as being between a little under 15% to a little over 18% of entire project effort. It is up to the project manager to understand this and other metrics, and to ensure that software “writers” are not given the opportunity to present their estimates out of context from the programme of work. Otherwise, their audience anticipates that a developers estimate is all there is and of course then blame developers for delays when their original estimates only accounted for < 20% of the project.

    Also, even the coding can be reasonably accurately estimated (within +/- 10%) by experienced estimators off the back of design specifications – just like engineers can estimate total cost of the back of engineering drawings.

    To my mind, the physical world with real people and real relationships, politics, changing environmental factors, project governance, etc is significantly more complex than software development – both of which I do.

    I mean, do you know it can take days to negotiate a sub-contractor’s price – and you can tell whether it will take an hour or a week until you finish the process. Yet, physical projects sometimes deal with thousands of sub contractors. And that is just one very minor aspect of a large engineering project. I’ll bet the negotiators would love to deal with something as predictable as code.

    Maybe my comments make a difference, maybe not; but the sooner the software industry pushes project management across all aspects of itself, the sooner our lives will be less about insufficient time and more about predictable outcomes for specific efforts put in.

    Cheers,
    Have a good Christmas.

  45. Good post and very true. What I’ve found is that intellectually everyone understands that early estimates are rough, but there organizaitonal/political/financial motives often beat out rationality.
    FWIW: I blogged a few days ago about how pert scheduling, the backbone of almost all project management techniques, never really worked the way everyone thought. If you’re interested in the illustrious history of project management fallacies, you might find it amusing.
    http://deathrayresearch.com/blog/2007/12/22/the-myth-of-pert/

  46. Pingback: Room for Sk3pticism » Stored links

  47. Patrick

    Enjoyable Post! Enjoyable in how resonant it is with the ludicrous and conceited stances of my programming friends. These fellows, some of them Carnegie Mellon and MIT grads, really thought they were going to be able to wake up whenever the wanted, work as much as they wanted, and be as obtuse as they wanted in any reply to the people they work for. Just a look at the bland slow responses they give anyone who actually nervously inquires about any completion dates. I have been bothered by the conceit and rejoicing at watching the walls close in on them as folks finally catch on to their game. The dot.com collapse and the introduction of actual working individuals from foreign markets started uncomfortably dwindling these social retards checks, and they are bitching more and more.

    The objection voiced in this post is not fundamentally to Project Management, as it claims. The general objection is merely to having to be asked a few questions…

    (1) umm, how long do you suppose it might take?

    (2) when do you think it might get done?

    (3) uuhh, mind giving me a ballpark estimate as to the cost?

    and, the dreaded…

    (4) how far along do you suppose you are?

    These types of questions annoy 14 year old boys, and of course, they annoy many programmers.

    Thank god for the increasing numbers of people with these skills, some of whom actually have a sense of what it means to work as a team.

  48. To say that Project Management is not full of people who are actually using it and abusing it is a blatant lie, but this is the same in almost every other profession (if PM is a profession). But to say that the whole thing is a hype is probably an overstatement. Project Management existed for thousands of years, but yet it was recently (relatively recently) called by its current name: Project Management.

    On the other hand, there are some people out there who really have the knowledge and the will to get things done, and that’s Project Management for them, it’s not about methodologies, it’s simply about getting the work done (to satisfy all stakeholders) in any possible way. That’s real Project Management.

  49. actualprogrammer

    Great article. I have a message for all the “project managers” out there: we are laughing at you, we are not doing what you think we are, you are useless and clueless.

  50. Every real IT person (programmer, network admin, DBA, etc) knows that PM is a total sham, a backdoor way into IT for people who can’t do actual IT work, scam artists.

  51. Pingback: Gantt charts don't work - Sandglaz Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s