A recent post by Mike on his blog Mike-o-Matic gave me flashbacks to some of the most ill-informed management decisions it’s been my horror to witness in the past. He provided 7 tips for the great build or buy debate which plagues IT departments everywhere. For those who don’t work in IT, this debate is basically “do we buy and off-the-shelf product to accomplish what we need or do we develop a custom tool in-house?” Again, if you don’t work in IT this might not seem relevant to you but I don’t think there’s a job today that isn’t affected by IT implementation decisions so this article might give you some insights into why certain IT systems that feel useless have been inflicted upon you.
There is not one single answer to this question. If you are working in a place that decrees there is a single answer (“we will always buy an existing product” – “we will always build our own solutions”) that’s your whole problem summed up there. If each situation isn’t judged on its own merits then it’s pretty much inevitable that some inappropriate decisions will be made. It shouldn’t be surprising that I have a bias towards up-front analysis – I am a Business Analyst after all – but I don’t recommend crippling progress with exhaustive (or should that be exhausting) analysis at the cost of actually doing something.
This being the angry blog of Mr Angry, I’ll illustrate my points with some insane decisions I’ve witnessed that have made me truly angry.
The first example I’ll use is a business that had a chronic problem with people developing a custom tool for every little task. I didn’t realise how bad the problem was at first because most of my work revolved around the core systems – propping up aging IBM AS/400 mainframes rather than developing any new tools. As an aside, I don’t even mention working on these dinosaurs of the Big Iron Age on my resume from fear a recruitment agency will try to place me somewhere else that uses them. I know some people like mainframes – I’m not one of them.
Realistically, it isn’t surprising that people were developing custom tools in this environment because the process for making changes to mainframe systems isn’t exactly nimble. The trouble is, the decisions being made were all short-term gain, long-term pain. One of Mike’s recommendations was to ban widespread use of MS Access across companies to limit people’s ability to make their own custom applications. He was being tongue in cheek but it’s a classic example of “a little knowledge is a dangerous thing.” People learn how to build their own little database applications so they run off and do it despite the fact it would usually be better to have a common tool used across the company.
In this company, people went one better (which really means one worse) and were building applications in Excel. People often stretch Excel’s ability (it’s a spreadsheet, dammit, that’s all it should be used for) to make it perform like a database (a bad idea as illustrated in this fun comic) but these people were actually building full-blown VB applications in Excel. For the non-geeks that means that although it looked they were running a custom built program, it was all jury-rigged on top of Excel. The data they were working with couldn’t be shared across the business (like it could have been if it was built on the mainframe as it was supposed to be) and it was pushing Excel way beyond its intended limits which meant it could collapse at any moment.
This was actually the problem I had been called to look at, their “solution” kept crashing and losing all their data. I heard they were running their work in Excel and though that meant they were running complex spreadsheets. I was absolutely stunned when I saw what they were doing. They had gone so far as to design a complex graphical user interface in VB and paste it on top – it looked like a fancy web application. “What can you do fix the problem?” It was a few minutes before I could think of something to say beyond:
“You’re the problem, my solution is to shoot you and put a hand grenade in your hard drive so nobody else copies your idea.”
In short, this was a textbook case of people developing custom solutions when they should have been looking for a standard company-wide application. The people doing this thought they were right because they believed they had solved their problem. What they had was a work-around, not a solution. The main problems with their work-arounds were that they weren’t available across the whole company (and a real solution was needed company wide) and they were totally unreliable.
For the other side of the coin I’ll use an example of a company that insisted on buying a system instead of looking at the possibility of developing a solution in-house. One of the problems this company had was they were trying to do too many big things at the same time and so management’s focus was completely splintered. The financial system, e-commerce engine and web publishing system were all planned to be replaced concurrently. Any one of these projects is a major undertaking but to plan to do all three at the same time is pretty goddam scary. Oh, and a new head of IT (Chief Information Officer/CIO) was put in place at the same time.
The new CIO issued a blanket ruling: “We are not a specialist IT development company,” (accurate enough statement) “so we will not develop any software in-house. We will always buy off the shelf. And we will never use open source, we will only use commercial products.” Ouch. That did not go down well in the IT department. The problem with his seemingly logical statement was that we might not have been specialist developers but there were some highly skilled and very motivated coders in the IT department. I was just the shitkicker BA but there were some damn smart people there.
It was my job to put together the requirements and tender documents for the web publishing tool so off I went and started. One day the lead developer comes up to me.
“Can we put in a submission for this project?” he asked.
“How do you mean?”
“Show the development team the tender documents all the vendors are getting and we’ll respond with our proposed solution along with everybody else.”
He outlined their proposed solution. They had a prototype that involved putting together a couple of open source applications to form a complete web publishing tool. They had worked out how to copy the existing kludgy database of web content straight across so there would be no interruption to the site. Once it was in this more stable architecture they could progressively tweak it until it was better structured which would make it much more manageable, allowing for the deployment of a consistent design across the site, easier editing and it would be much easier to manage the growth of the site. This solution would be quicker to deploy than any alternative, easier to maintain and much cheaper than any alternative.
Management responded with a wall of silence. The wall was occasionally broken with the rote “we are not a development house” line but largely there seemed to be a mentality of ignore it and it will go away. This proved to be an effective strategy as all of the motivated developers quit over the next six months and so did I. So the CIO’s vision was not challenged and the company missed a huge opportunity.
At the end of the day, this is not a decision to make lightly. As Mike pointed out, software developers are often the highest paid non-management staff in a company and having them work on development can be costly both in dollar terms and lost opportunity terms. Uncontrolled custom development usually ends up with a series of incompatible (and often unstable) systems across departments that cost time and money to maintain and as often as not, this aspect causes reduced efficiency, not the improvements each individual claims. But likewise, arbitrarily ignoring the possibilities of in-house development can cost the company more and can definitely affect the morale of the IT department (an issue that management ignores at its peril).
The following list is a summary of Mike’s 7 tips. They are a good guide on how to make the best “Buy or Build” decision:
1. Do research before building requirements
2. Be willing to change your behaviour to suit a software tool
3. Consider open source
4. Get a range of people involved in evaluating options
5. Focus on flexibility
6. Remove the incentive to empire-build
7. Ban people from building custom applications in Microsoft Access (really, really ban them from building custom applications in Excel)
These are not necessarily in order of importance but number one should always be first as everything else flows from this. In short, THINK before making a decision.
Angry 365 Days a Year 
Yet another diss on Excel, a reliable application that actually works and that the average user can understand (as opposed to Access, or “open source”). A good consultant would have told them that it was a good prototype, or even better fixed the problems, and then used it as a basis for a proper application.
This is the kind of attitude that IT used to have and caused Lotus 123 to take off, because users were able to develop stuff that IT wouldn’t. Going full circle I expect, lock down the PC and close out the users and then get in shock when they do things on their own.
I have a tendency to agree with Oliver regarding Excel and, as I’m sure you could guess, would extend that sentiment to Access, I think that there are other issues. When IT doesn’t have the resources to do all the work, then the users need to do something other than wallpaper their offices with sticky notes. What would you have them do? Sure, the stuff can turn out to be absolutely horrid, but at least you have a functional spec and data in a format that has enough structure to be importable to a real system.
What we’ve done where I work is to give *every* user Excel training. Now they know that every spreadsheet *starts* with a raw list of unsummarized data suitable for use with a Pivot Table. Yes, we actually spent the hour it takes to teach someone to build and use a Pivot Table. Along the way, I’ve managed to convince most of them to call me before they start so that I can give them a bit of advice. I’ve identified a few candidates for some training in basic database theory and practice and hope to teach them to use Access, if not wisely, at least not dangerously. The idea is to build an information systems pipeline that starts with people solving their own problems in maintainable ways (Excel), goes through some clean-up and refinement and getting wider distribution (Access), and finally makes it to me for final application development and integration (currently C# on SQL Server).
It’ll only take a few years to find out whether this works for us and there’s no way it can be as bad as what has been going on for the last decade or so.
But to move back to the topic at hand, I whole-heartedly agree with the other stuff. Escpecially the bit about thinking first.
Salamaat,
yet another example of the madness in the IT industry…at first i didn’t agree with oliver; but on second thought he does have a point. Short of locking users out completely, i guess IT-ers would have to suck it up and deal with the occasional/not so occasional catastrophes they create.
meh.
I’m not sure I made clear what people were actually doing in Excel… They weren’t using it as it was intended (learning how to use pivot tables can be incredibly valuable), they were building VB APPLICATIONS in it. Excel is nowhere near robust enough to handle this reliably and they ended up making their own lives a misery when their apps crashed and they lost all their data.
I did make a point of acknowledging their frustration with a system that moved at a glacial pace but making bad decisions regarding workarounds creates a worse situation, not a better one. And doing work that’s supposed to be distributed across the company on a custom application that exists only on your hard drive is not exactly efficient or trustworthy.
The problem of users not having resources to do their work effectively is a different and important one. The problem of a totally unresponsive and arrogant IT department is also a serious one. But if you attempt to deal with this by making bad decisions that result in incompatible applications and data structures (which is what happens if people develop custome apps in isolation) then you become part of the problem and you actually end up further away from having a real and lasting solution.
It’s like scratching an itch too hard – it feels really good at the time but will cause you pain down the line.
I know you were talking about Excel VBA apps, but I let my own anger get the better of me. I’m frankly tired of my colleagues’ “solution”: ban Excel and Access. Why not put some resources into training, guidance, and support for Excel and Access? Sure, we only have 1 or 2 people outside of IT that should have Access and maybe a couple of dozen that should do serious Excel work, but I’m amazed at how little it takes to keep these folks on the straight and narrow. They don’t do VBA and nothing goes live without IT inspection.
We’ve only just started down that road, so it’s impossible to declare either success or failure, but we are quite confident that we won’t destroy ourselves as a result. Early results (6 months) are encouraging.
That’s OK Ron, I know I was joking and Mike was as well in his original post. I took the side of programmers this time because I thought I’d been picking on them too much lately 🙂 I think I need to address the issue of communication between IT, users and management – it’s a real “root issue” of problems that spill out in all sorts of ways.
If the needs of each party aren’t being met fairly of course people go for work-arounds. I probably should have also pointed out that the workplace I was using as an example in this case was by far the most dysfunctional place I have ever had the misfortune to be trapped in. The strategy you are describing seems eminently sensible – I hope it goes well for you.
Uhmmm… the AS/400 is not a mainframe….
Rick: I’m not a techie 🙂 Feel free to describe the difference (seriously) I was just using my ill-informed layman’s description.