Via today’s Ottawa Citizen, this article: Aurora update soars $132M over budget: Aircraft upgrades began before DND knew extent of project
The CP-140 Aurora is a reconnaissance aircraft that has been used for over twenty seven years by the Canadian Department of Defense to patrol our borders – for the past nine years they’ve been undergoing retrofits to update sensor arrays, avionics and computers which, to the shock and dismay of a Government auditor, have gone $132M over budget:
The program to upgrade the CP140 Aurora's computers and avionics began in 2002 and was supposed to cost $197 million. But that figure has ballooned to a little more than $329 million, according to a newly released Defence Department audit of the project.
The audit, completed in August, found that the modernization program was difficult to manage because the Defence Department didn't know the full requirements of the project until it was well under way.
Of course they couldn’t know that! Who could? The author and auditor seem to suggest that there is a great human failing when contractors’ powers of clairvoyance fail them! As we get further in, the full spectre of the project’s failure is revealed:
"Since the CP140 modernization program was incremental in nature, the full requirements were not known until year four of the nine-year contract, resulting in amendments that made it difficult to manage and invoke the appropriate terms of the contract," the audit conducted by the department's Chief of Review Services states.
I’m not so sure about what’s meant above about the connection between an “incremental in nature” program leading to feature/scope creep that blew out BDUF estimates – I’ll have to assume that we’re looking at a series of waterfall projects, each compounding the problems of the other.
In any event, this type of fail-late exercise should be immediately familiar to most folks in IT/software development. It’s what happens in 90% of your projects.
By June 2005, the amendments to the contract, which included the need for training simulators and the integration of additional sensors, boosted the contract's value 67 per cent over the original estimate, the report noted.
Because the audit on the Aurora data management system contract is so heavily censored, it is difficult to determine whether the costs are still climbing. The project is to finish in 2009.
Again, if you’ve been on more than a few enterprise scale projects this will be immediately familiar to you: The “churning” that results from gathering new requirements, implementing them a la waterfall and finding out that they’ve introduced bugs requiring more gather-code-fix cycles.
Moving on, here’s where the matter gets a little audacious vis-a-vis the audit:
The audit was not designed to assess the performance of the companies involved, but was focused on the department's processes and practices. The names of the firms working on the Aurora data management system contract have been censored from the document.
Why not assess the performance of the companies involved? Are they failing because of their own flawed practices or is it a result of DND’s project requirements, which I suspect bear more than a passing resemblance to the U.S. DoD STD 2167 project stipulations from over 36 years ago – for the uninitiated, they specify all projects to be executed in a single-pass waterfall process.
Unsurprisingly, the auditor comes to the conclusion that it’s nearly impossible to assess true ROI (return on investment) – but not for reasons of the process, but rather that the amendments weren’t put out for bid!
The audit notes that some of the amendments to the Aurora contract were awarded without competition. "Value for money assurance cannot be provided for amendments worth (censored) because they were sole-sourced to the prime contractor," it added.
Balderdash. If you farmed these amendments out to other BDUF/waterfall contractors, you’d be just as likely to have even more compounded problems as teams who are unfamiliar with the project need to ramp up as part of their execution. The problem is not with sole-sourcing. It’s with the process employed. ROI cannot be realized when projects are integrated and tested so late in the delivery cycle.
I find it both amusing and frustrating (in the manner of continually hitting one’s thumb with a hammer) to see failed BDUF/waterfall thinking so pervasive – some try to duff it off as “the confluence of many factors” contributing to failure. True, but why not then remove the #1 factor that almost assuredly guarantees failure: BDUF/waterfall?