Laziness can be a hole that's hard to climb out of.
I've written before about the Confluence automated build system; a typical exercise in Postmodern Programming. What's the most convenient way to shove files around and launch programs? Shell-scripting. What's the most convenient way to do text processing? Perl. What's the first useable script I can find on the net to do MIME multipart emails? Python. The result is a bit of a mess, but it works.
Of course, it gets worse. Now we're running the functional tests against multiple databases and multiple application servers, the scripts are even more complicated, and the results verbose enough that they have to be summarised for the email, and web-pages created for the detailed results. This roughly translates to "more Perl".
One problem with this approach is, of course, that I'm the only one who knows its workings enough to maintain it.
Another is that it makes it hard to integrate the build with more sane systems like CruiseControl. The lure of something that "works well enough for now" is quite strong compared to the rather painful thought of something that we might manage to make perform the same tasks, given a week of hacking on the test-harness and writing Jelly scripts.
The sad thing is that I thought when I started writing this that the process of writing would lead me to some insight or conclusion, some way out of this mess. All it's resulted in, though, is an aimless ramble.
That's probably a bad sign.
And there is an entire circle of Hell reserved for anyone who suggests Groovy would improve things.
Update: Here's a nice sample of the mail the build currently sends out. Ignore the failing unit tests, they're, er... They're just there to demonstrate what a failure would look like.
Have you considered using Groovy?
Whenever I get into this kind of situation, the question I ask myself is, "How much do I care that everything might break if I take a holiday?"
PS: Will there be weeping and gnashing of teeth if I were to utter a word that starts with "Jy" and ends with "thon"?
I think that Python could cover the tasks you are using shell scripting and Perl for fairly well. At least then you would remove those dependencies and have build tools and scripts in only one language.
Anthony: I'd love to rewrite everything in the one language: preferably Ruby, or maybe it'd be a neat excuse to learn more Python. But doing would just make the "Only Charles can maintain the scripts" problem worse.
The reason that the scripts are "mostly shell with some Perl" is that about matches the chance that someone else in the office will know enough to be able to decipher them. :)
Ant? I have built quite complex build systems in nothing but Ant (plus a weeny bit of JACL to control automated websphere deployment). We deliver a whole bunch of EARS WARS and JARS to production staff ready for them to add the necessary configuration, then press 'go' for deploying to webpshere.
Well, it's a sign that you're not seeing the forest for the trees, frankly.
I have yet to find a build process that cannot be done with ant + custom tasks. Your deliverable is just a damn war, how hard can it possibly be?
Building the WAR is the easy part. If all I had to do was build a war, the build script would be: "maven confluence:war"
I have to build a WAR that passes a suite of functional smoke-tests against various combinations of Tomcat, Resin, Orion, Weblogic, HSQL, MySQL, Postgres and Oracle. I need to do it automatically every hour of the working day, and I want notifications delivered in a nice, clear traffic-light "Have we broken anything?" format.
(I've linked to an example of one of these emails up in the article, if anyone's curious)
And that's what I've got right now. It's just rather ugly because one of the requirements for writing it was "get it working quickly so you can go back to coding on the product."
Ultimately, when I am possessed of that luxury known as time, it will end up as a bunch of Ant tasks (or more likely Maven goals) so that we can run it all under CruiseControl.
What we have now is Good Enough that any attempt to replace it with something sensible and homogenous would involve a significant amount of work modifying the test system and writing custom tasks... and leave us (functionally) exactly where we started, just with a whole lot more angle-brackets.
Well, then I for one applaud your stance, just as long as you're aware that that argument applies to 'wasting' time doing major refactorings, or going away to update broken functional tests, etc etc.
Good for you though for focussing on the deliverable and not the 'cool' stuff around it!
I think the reason why scripting stuff goes into a complete mess is because we don't treat them as real software. We patch a little here, a little there, add some stuff here, tweak this then that. After a while the little bash script we started with three months ago has turned into a real system with a lot of functionality, but what a mess it is!! Adding that last bit of crucial functionality turns out to be a complete nightmare and ends up breaking the whole thing.
There's no silver bullet to solving this, but there's two things that will definitely improve the situation:
1) Simplify your codebase so that the build system doesn't has to be so damned complicated. (Easier said than done when you're already in the mess, but try to think about this from day one.)
2) Treat the scripting system as a real system; I'm not saying you necessarily have to do TDD but you should definitely design and refactor it.
Lastly, I just have to add a plug for my beloved Ruby. Ruby is perfect for doing large quite complicated scripting systems because you can do clever text-processing things in it like Perl but you can also structure your code in a nice object-oriented way. It's not gonna solve your problems, but it's going to make it quite a lot more pleasant.