When a project goes off the rails

One of the hardest things to do in any IT project (and for that matter, probably any sort of project) is to face up to the fact it’s gone off the rails. Things have gone wrong. It’s time to stop the madness. There are a range of reasons it can be hard to face up to this: fear of failure, inertia, being so busy with details that you lose sight of the “big picture”, corporate pressure to keep going no matter what.

All the reasons and variations for failing to acknowledge that a project is in trouble usually have one thing in common: denial. Many people seem happy pretend that so long as nobody says out loud that the project is in trouble then there isn’t actually a problem. Nobody want to burst the illusory bubble for fear of being branded negative or, worse still, being targeted as the source of the problems (some twisted variant of “first who smelt it, dealt it”). The denial of the obvious is so widespread it’s a wonder that heads aren’t exploding from cognitive dissonance in IT departments all over the world.

I was working at a large construction company recently to help them procure and deploy a project management software tool. The company was rapidly expanding and was managing about $200 million worth of projects each year. The paper based processes that had worked fine when the company was smaller (around $40 million per annum) weren’t scaling well and the pressure was starting to tell on everyone involved.

If someone in management asked the type of questions people in management like to ask (pesky things like “Is the project on schedule?” “How much money have we spent?” “How much money is still in the budget?”) the short answer was “nobody knows”. The longer answer was some poor bastard spent about a week out of every month chasing pieces of paper that were filed god-knows-where and desperately calling around to find someone who might know the answer.

In short, you didn’t want to be involved if there was an audit at this company.

So the decision was made to go to market and find a solution. All of the big hitters in this market responded to the tender (thankfully we didn’t go with the German company whose acronym rhymes with CRAP – I would have quit if they’d won the tender). The early ball-park estimate for defining requirements, sending out a tender and picking a winner was 9 months. It ended up taking 12 months to finalise but that’s pretty damn accurate for a ball-park estimate.

The initial estimates for getting the software configured and deployed was eight months. Two months into the design phase the timeline had been pushed out to 12 months but nobody was panicking. At the three month mark I realised that while maybe panic wasn’t called for, we needed to stop what we were doing and seriously re-assess our approach.

One of the positives of this job was that I was generally treated very well. By that I mean I was treated like a professional. My experience and expertise were respected. I was actually listened to. Most IT people who have worked in non-IT environments (especially something as nuts-and-bolts as construction) will tell you how rare that is. This had a kind of downside in that I was the “lead analyst” and it was down to me to make hard calls like “we’re going off the rails.”

Hard as it may be to believe from reading this blog, in a work situation I don’t really revel in being the centre of attention. And saying that there are problems with a multi-million dollar project is a great way to REALLY be the centre of attention. From this and other experiences I have learned a few things about how to be the bearer of potentially bad news. So here are Mr Angry’s tips for dealing with a project that’s running off the rails:

1. Voice your concerns sooner rather than later Trust your instincts – if you think something’s wrong, it probably is. Problems rarely fix themselves magically – they usually get worse if left unresolved. While it can be daunting to admit to having problems it is usually far easier to fix them early when they’re first making themselves felt rather than later when they’ve spiralled out of control.

2. Have a plan There’s identifying problems and then there’s complaining. One platitude I’m a big believer in is “if you’re not part of the solution, you’re part of the problem.” If you’re going to say there’s a problem you should be proposing a solution or at least a strategy for reaching a solution. At the very least identify clearly why you think the project is running off the rails. Don’t say “this sucks,” say “if we don’t address this problem we’re going to face nasty outcomes x, y and z.”

3. Accept responsibility when appropriate I’m not saying be a sucker or be everyone else’s whipping boy. But I am saying it’s transparent and bloody annoying when someone sprays blame in every direction while claiming total innocence. If there’s something you need to cop to, then cop to it. But this is a situation where you really need a solution ready. Be prepared along the lines of “Look, I didn’t catch this earlier so we’ve actually been developing the project in the wrong direction. But if we do this and this then we can get things back on track.”

4. Don’t let the problem be ignored In many cases, when you report an issue the response from management can be along the lines of “That isn’t your responsibility, just do your job.” The following piece of advice is aimed at anyone and everyone who considers themselves an I.T. professional: you’re a professional. You have a professional responsibility to not allow important issues to be swept under the carpet. Plus, you need to cover your butt. Don’t kick and pout and scream but if you’re being told to let something go, DOCUMENT IT. Whether it’s the minutes of a meeting or an email to your manager, make sure your concerns as well as the instruction to let it go are clearly documented.

5. Recognise when the problem is institutional There are cases when the problem simply isn’t going to go away and isn’t going to be dealt with effectively. The workplace is institutionally dysfunctional. If you’re stuck somewhere that punishes people who identify problems then you have to accept that reality. I recommend a two-pronged course of action: first, cover you butt (see point 4, above). Second, GET OUT! Seriously, this type of workplace damages your mind and soul.

Making the call that a project has gone off the rails is rarely going to be easy (especially if it carries the connotation of “we’re pretty much going to have to abandon the last year’s work.”) While the Chinese don’t really use the same word for “crisis” and “opportunity” (crisitunity!) in a perfect, or even pretty good, world identifying a problem is the first step to solving it. If you or your workplace are to scared to face up to the reality of a project that’s run off the rails, all I have to say is enjoy your death march.



Filed under Work

13 responses to “When a project goes off the rails

  1. I’ve made enemies and lost jobs just for speaking my mind, but I sleep like a baby every night. In my humble opinion there is no job worth losing sleep over (unless, of course, it’s because you’re busy).

    I agree completely with you Mr. Angry. If things are fucked you owe it to yourself to be honest.

  2. Dear Mr Angry! How are you? I was reading through your list and it occurs to me that it is also an appropriate list for “rules of life”! Though for number 5, we could substitute “recognize you are human and ensure you have you sleep for 7 hours a night”

    Nice to see you. I have yourtubing quite a bit and found my favourite show from way back when “Alo Alo” – have you heard of it. As for the law suit against google/youtube – I have my strong opinions about that, as you can imagine.

    Glad you kept me on your blog roll.

    Look forward to seeing you on my blog space!
    Keep well!

  3. kyklops: exactly, if you aren’t true to yourself then you have no right to expect other people to look after you.

    Dr Nazli: Welcome back my dear! I’m glad you find my tips have wider application. Oh, and I don’t desert my friends just because they’re busy for a little while 🙂

  4. Pingback: Top Posts « WordPress.com

  5. Monkeyget

    ” Many people seem happy pretend that so long as nobody says out loud that the project is in trouble then there isn’t actually a problem […] The denial of the obvious is so widespread it’s a wonder that heads aren’t exploding from cognitive dissonance in IT departments all over the world.”
    That’s a really interesting point. You just do your work and you don’t (want to) realize that you’re doing something wrong/bad.

    The same point was made by Alex Papadimoulis
    “[…]Instead of realizing that the first applications they wrote were poorly designed, poorly programmed, and soon-to-be failure – like everyone’s first are – they chalked them off as a success and moved on to the next project. They developed more software. Bigger software. More expensive software. Wrong software. All because they kept thinking that the completion of the project meant that it was a success.

    Had they simply admitted to earlier failure – and learned from it – they would have never created such an epic disaster. By imagining that their past projects were a success, they had done far worse than failure.


    That brings me to the second advice of the book “The Pragmatic Programmer” : Think about your work. If you don’t think about what you’re doing and more particulary what you’re doing wrong then you can’t improve.

    That’s why I tend to be pessimist. I start from the principle that I suck, that what I do equally suck and that what i’m doing is going to fail. That forces me to improve what I’m doing constantly. It also has the nice side effect that things generally turn out better than anticipated 🙂 .

  6. Ah yes, promise low and deliver high. Everyone should do that, pleasant surprises are always better than nasty shocks.

  7. Great thoughts mr. angry, keep it up

  8. Oooh. Where to start? I’ve a situation where we have a Great Big Hairy Dependency on an update to a system which is estimated to be delivered at the end of the year. Now the only sane thing to do with this system is the Ripley Option – take off and nuke the whole site from orbit. If you aren’t going to do that, then the next best thing is to:

    1) Talk to the users and gather business-, process- and workflow-requirements
    2) Re-engineer the fucking processes ok?
    3) Make it easy to use

    This is a to-do list in order of importance not an options list.

    Unfortunately this project seems to think that they can replace all of the above with (4) make it pretty. If it’s pretty, you see, we won’t notice it’s shite until it’s too late. As it was described to me today “it’ll be a turd covered in glitter”.

    This is a project in need of:

    1) A sponsor
    2) Money
    3) A steering group (not just a list of “workshop attendees”)
    4) Documented requirements (not just a list of “outputs” from the “workshops”)
    5) A functional spec (not just a series of paintshopped “screenshots”)
    6) A technical spec (any fucking technical spec – the one for designing the hovercraft would be better than nothing)
    7) Someone, anyone, with balls and integrity.
    8) A great big banner saying “It’s the processes, Fuckwit. It’s not the front end It’s the frigging PROCESSES.

    We can’t talk to them, because they have already gathered their requirements, (a year a-fucking-go), and they’ve been signed off (by the previous project manager’s mother, her hairdresser and her ickle wickle puddy cat so far as I can make out). We can’t actually – like see the requirements. Oh no. They know what the requirements are. They’ve been signed off. What do we know? We’re just users. And here we go round in circles again.

    The fact that we have a Great Big Hairy Dependency and represent 35,000 users is irrelevent. They have their fingers stuck in their ears, their eyes are screwed shut, and they are singing “la la la la la” at the top of their pathetic little voices.

    But it’s ok. I shouldn’t be worried, and I shouldn’t be nasty.

    They did workshops. It’ll all be fine…..

    Thank you.


  9. Aphra: On the plus side, you know what to do. On the nagative side, it sounds like you’re pushing shit uphill. Good luck! Plus, I love how you summed it all up with “The Ripley Option”

  10. 😀

    It is at least shit covered in glitter, so maybe it’ll be the glitter that rains down on me, not the shit.


  11. Pingback: Best of Feeds - 21 links - blogging, development, twitter, blog, web2.0, socialsoftware « //engtech

  12. Pingback: book » When a project goes off the rails

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s