The upcoming 1.6 release of the open source dispersed storage network software will see a number of usability enhancements. Foremost will be a vast simplification of software configuration. The 1.1 release had a number of fairly unintuitive XML configuration files. For the next release, for each package there will be a single configuration file that needs to be edited.

We will have a “three-tiered” configuration system consisting of:

• Tier 1: Central configuration (the "registry") for all things globally relevant or required to be configured centrally. The open source release will provide a consistent set of CLI tools to manipulate the registry. 
• Tier 2: Local administrative configuration. This will be a simple name/value “.conf” file. This file contains only settings that the typical administrator will need to change. It will not contain unstable or new “tweaky” configurations. The rule of thumb will generally be “is there a reason 80% of the user base will want to change this option?” If the answer is no, it will not be configurable in the ".conf" file. Most configuration options will contain defaults that should be sufficient for most people.
• Tier 3: Advanced module configuration. For module-specific configuration that is "new", "unstable", or "tweaky". Also for configuration that requires some structure but is not important enough to write a CLI for. Most of these advanced module configuration files will be XML files, but the exact format will be module specific. Most new options will be initially introduced in advanced module configurations.
  

All options in the first two tiers are expected to be fully documented before releases. We will want to be especially careful to avoid making backwards-incompatible changes in them between releases, especially in the first tier.

The third tier will likely not be fully documented and any changes may be incompatible between releases. If you don’t provide advanced configuration options there will be defaults compiled into the binaries that will be used. If you do provide advanced configuration files you, as a user, will be responsible for making sure that change them appropriately during an upgrade as there will likely be required values that you have to add or old unsupported values that you will have to remove.

Note that some modules will have their own internal “state”, which is different from the three configuration tiers mentioned above. For example, the slice storage module contains a database that keeps track of mount points and the slice servers and vaults stored on them. The administrator will manipulate this database when he or she adds mount points to the slice server, but the main point of the database is to keep track of the internal state of the slice storage module. Events like vault creation and deletion and slice I/O will cause database modifications. Upgrades to this database occur when upgrading to a new module (for the next few releases this will probably be a manual process).

On the forums, brokebit asks “where is documentation posted?”  A timely question, because this week we’ve spent some time planning what documentation we hope to deliver for the upcoming 1.6 release.

In the past our documentation process as looked somewhat like this: Our developers jot down some notes in various places, and with luck we get them to post those notes to our internal wiki. Over time use of the wiki has generally improved so there is pretty good information there now. We use MediaWiki, which in our setup is basically impossible to search. So finding the documentation is hard. But that’s a topic for another day!

So, when it comes time for the actual commercial release, we’ve got a bunch of good resources lying around. We have one of our business analysts  compile all of this information and hand it off to a consultant that does our training materials (we have an extensive training program for our resellers and customers) for him to compile into a complete package.

In the meantime, our open source release has a text file containing some basic installation instructions. So this is where John and I are focusing our efforts for the next release. Vicente Cano, our build master, has proposed that we try to ship 1.6 with the following documents:

• Installation Guide
• Admin Guide
• User Guide
  

We haven’t quite hashed out exactly what these will contain or what they will look like. We have been looking to Splunk, MySQL, Apache HTTP, and the Linux From Scratch Project for inspiration, but any feedback on our ideas would be appreciated.

Even while we’re trying to figure out what we’re going to publish we need to figure out how we’re going to create the material. Our entire company is very focused on getting our commercial release out the door and we don’t have a lot of time to expend on open source documentation. This means that we need to be able to collect information from as many people as possible in a very lightweight manner.

Getting our developers to write little bits of documentation on the parts of the software they have written is potentially doable. If we were to ask for this we would have to first determine what format we wanted them to write in. They are already familiar with Microsoft Word. Word is what we use to compile our commercial documentation. I think that Word is by far the best writing tool our industry has to offer. It has incredible change tracking tools, grammar correction, and all the other drafting tools a technical writer could dream of.

Unfortunately, it is extremely bad karma to develop open source software documentation in Word—I don’t know if I could sleep at night if we committed such files to our repo. My main problem with Word files is that they are binary (even the new WordML is saved as a ZIP archive) so proper SVN diffs are impossible. They are also unstructured, so combining them into a nicely organized book is sort of an inexact science of various style combinations. You also can’t use Word files for single-sourcing very effectively.

For the long term Vicente and I generally think that DocBook is our best bet. DocBook is an open standard, supports structured documentation and single-sourcing, and has good transformation tools for HTML and PDF. The major downside is that pure DocBook has a steep learning curve. The ideal solution would be to find a good tool for Microsoft Word to DocBook round-tripping. I’ve looked long and hard and I haven’t found any really good solutions. There are some good DocBook editors out there, but any editor is going to have a learning curve and will not be as full-featured as Word.

So we haven’t fully solved this problem. In the meantime I think that we’ll have people write in text format. They can draft in Word if they want but since we’ll have to commit something reasonable to the repo it might as well be text. To make it a bit easier, we can use AsciiDoc, a text markup similar to StructuredText. This will allow developers to write in a very natural format that can eventually be converted over to DocBook when we move further in that direction.


As a final note, I found a short article about how non-programmers use documentation that seems very insightful. I would be interested in any feedback as to what sort of documentation our fledgling community is interested in.