The causes are not confined only to Governments, nor to large projects: but the effects scale up, so what will cause an over-run in budget on a small project will cause an utter disaster on a big one.
I'll paraphrase and simplify them:
- Not consulting with stakeholders - those who have a vested interest in getting a system that helps them. Too often, requirements are specified by customer personnel who won't use the system, and don't really know what it should be doing. You can even get a bun-fight between different stakeholders with contradictory requirements. Get these over with, or at least identified, or you'll fail.
- Being insufficiently flexible and unable to learn what is really required - or even possible - during development. In the course of building something, great hairy unknowns become better and better understood, and things which were thought to be easy might be determined to be hard, or worse, both impossible and irrelevant, un-neccessary. Something that will break budgets to provide, but which the customer doesn't actually want. Except the contract calls or it.
- Lack of incremental progress reporting and incremental funding no more than one step ahead of progress. There's a temptation to take a failing project and keep on developing until the money runs out. Projects that are doomed from an early stage should be shot to put them out of their misery, and not kept on as festering and rotting undead, shambling on till they finally disintegrate.
- "Just a little bit more" - feeping creaturism. Creeping featurism, where additional requirements are added on, with no additional budget. Sometimes it's better to develop a system that works - but doesn't do everything - than not deliver a system at all. And then upgrade in a planned way with further releases. A good architecture will have a lot of "fitted for but not with" flexibility, a foundation that will take a 12 storey structure even if the spec only calls for 8. Just in case.
- Doing what has always been done, but with computers, rather than analysing what should be done, and implementing that. The world is full of systems that replicate pre-computer methodology, with all sorts of bottlenecks and inefficiencies that were unavoidable when using pen and paper, but now are just embarrassments.
- Setting to work costs not being budgeted for. An example would be data cleansing - making sure existing databases have good data in, or you'll just be delivering greater and greater quantities of used food. And making sure that the parallel operation where the old system is gently phased out and the new one gently phased in is planned and budgeted for.
As they say in the Classics, Read The Whole Thing.