RSS Feed
Atom Feed
Re-writing vs. Modifying Software
You may have noticed that the open source release we posted last month was a complete re-write of the code based we published last year. Not a single line of code was unchanged. This change wasn’t just due to the change from C++ to Java, but was a result of a decision to completely re-architect, re-design and re-write the Dispersed Storage software.
I’ve spoken a few times with Joe Jablonski of Acumence and others about the merits of re-writing software vs. enhancing an existing code base and Joe and I agree that software development organizations too often make the mistake of continuing to enhance weakening code base vs. re-writing from scratch.
I think this misjudgment comes from a fundamental misunderstanding of the nature of modern software development. Modern software development tools in the hands of capable developers can quickly produce complex software. (We are using SCRUM as our development methodology, by the way, and have been quite pleased with the results.) But software development does not just mean the act of writing software. And the time required to write complex software is not just the time required to type the code. Completing a complex software development project requires dynamic coordination of requirements definition, architecture, design, development, testing, validation, tuning, and enhancement. If done correctly, the act of writing code is only a portion of the time and effort required for software development, especially for complex software and especially for a new type of complex software.
Our goal in the initial production release of Dispersed Storage software is to create an outstanding software foundation on which we and others can build Dispersed Storage solutions. The work we did in 2005 and 2006 provided many insights in how to build a Dispersed Storage system and that know-how enabled us around the beginning of this year to know that we needed to re-write our software. That know-how also included knowing how to proceed toward an outstanding initial production release.
Whether we realize that goal will ultimately be determined by market acceptance of our software and specifically whether it provides the reliability, security, performance, scalability, longevity and cost-effectiveness benefits we envision. But the preliminary results we are seeing now from our re-write over the past year so far exceed last year’s results that we know that re-writing was a necessary step.
re: rewrite vs modify
Fortunately for Cleversafe, the decision to rewrite paid off. Of course, I think you and I can agree that such a decision would have been far more costly and difficult [certainly more painful] down the road with a larger install base and more moving parts. I'm glad you decided to tackle the project upstream while you still can.
It'll be interesting to see if the company is as willing [and able] to rewrite the code base once a substantial number of users have developed application- and system-specific dependencies on it. Cleversafe will then have to take into consideration the potentially costly impact on thousands of customer environments as well. It's certainly what keeps Microsoft engineers up at night.
On a different note...
One of the dangers of software development lies in the psyche of engineers and their tendency to believe they can build a better mousetrap than their predecessors. Obviously, this isn't always true. I'm particularly wary of engineers who carry the dangerous [design] elegance or [platform] bias genes.
In that context, I'm curious about Cleversafe's decision to move from C++ to Java. Would you provide readers with some insight into the design requirements that motivated Cleversafe to move away from C++, and, of the available alternatives, to choose Java? I'm sure they'll find the insight helpful when they face similar requirements in their own projects.