The story about our Wall Street Journal Innovation award went on Slashdot last Saturday. The post is in the form of an “Ask Slashdot” where the poster wanted information about easy distributed storage and backup solutions. This is a Slashdot perennial. I found many of the comments to that post very illuminating.

Many of the comments found our solution to not stack up well against existing software that tech savvy people deploy for personal, family, or small business use.  We certainly agree that our software isn't entirely ready for that audience, largely because of a lack of usability features. We’ve been thinking a lot recently about what we need to do to make it better.

The first problem to tackle is that of our target audience. As a company we started out thinking we would go after the consumer storage and backup market. However, this post was spot on in saying, “as for Cleversafe, the idea is as old as forward error correction, but the economics and management never seem to quite work out.” As a company it is the management of a large distributed storage network that will define our commercial offerings.

Unfortunately, for any casual user with a very limited amount of data and a low supply of bandwidth it doesn’t seem to make sense to set up and maintain the large amount of hardware that would make our solution advantageous.  For example, a basic configuration we support for our commercial product is the “8/6” vault. This means that data is split among eight different machines, any 2 of which can go down before the data is rendered inaccessible. This is fairly modest and yet would require that you have at least eight machines, which, at least for me, would be a tall order to set up at home.

This isn’t to say that a storage network won't ever be applicable in these environments. The core benefits of security, reliability, and performance are tremendously attractive to everyone, and to every backup scenario. Right now the really significant savings kick in when you have vast amounts of data to store. Our commercial customers typically have at least 8 or more terabytes of data.

So what about the “mom-and-pop” consumers out there? We seem to see a lot of new backup technology cropping up all over the world. This post mentioned CrashPlan, a cross-platform backup product written in Java. I love CrashPlan and just bought a CrashPlan PRO license for my home network. Another Slashdotter mentioned Dropbox. I haven’t personally used Dropbox, but from what I’ve seen it looks great. I think that products like these are the future of consumer storage and backup. They are generally simple and easy to use. They don’t have all the bells and whistles that products like NetBackup have, but they get the job done and it’s pretty hard to set them up incorrectly.

So how does the Cleversafe dispersed storage network fit here? What the consumer market lacked before these products was an intelligent, easy-to-use interface to accessing your data.  While these new products fill that void very well, we feel they don't offer much innovation when it comes to actual storage of your data. We want to develop a dispersed storage “platform” that companies can use as a better back-end storage solution so they can focus on innovation at the front-end.

We'd really like to see an increase in consumer-oriented deployments, and we'd love to help out in making this happen.  We’d really like feedback while we work towards refining our open source project. What features do you feel are lacking in our current release that would assist you in this style of deployment?  We'd really like to get a feeling for how this works out and what features are needed, so that we can begin to prioritize development to better serve this end.

If you’re storing a large volume of data you should definitely give our dispersed storage network a try. If you’re interested in storing personal data, or data for consumers, our open source solution might be a good starting point for you. We'd love to hear from your experiences in deploying our software in more consumer-oriented settings, and if you ever need advice or assistance in setup of, please feel free to contact us directly.

On a related note there has been some interest in a setting up a “cooperative dispersed storage network” that our community can use for testing and development. We're working on details of implementing such a network, and will provide details in future posts.

Wesley Leggette and I are very proud to announce that we have been named Open Source Community Managers for the Cleversafe dispersed storage project. Our history with this project goes back to its very roots, and we've had a steady hand in driving forward the initial open source strategy and decisions. In our close work together over the years as core developers, Wesley and I have established a strong and successful rapport, and we really look forward to the opportunity of applying ourselves to the continued improvement of our open source interactions and policies.

Cleversafe's decision to name us as Managers is a testament to the company's fundamental belief in the value of cooperative experimentation and development in an open environment. This is a value deeply rooted in our daily lives, both with the technical and business staff at our company. This decision also signifies our project's maturation. Up until this point, it has been all hands on deck to implement, test and deploy our first version of the dispersed storage network, and as such we haven't had the opportunity to fully dedicate ourselves to fostering innovation in our open source community. As Open Source Managers our prime directive will be just this - to tend to the care and feeding of this project, and those who wish to join us.

In the past, Wesley and I have been involved at various levels in open source development projects and users groups, both at the international and local community level, and our combined experience with these has given us a precise understanding of how best to foster this community. With no further ado, we're going to begin execution of that vision. We're deprecating usage of the current forums, which we've found to be difficult to use in carrying out technical conversation. In it's place we've deployed a bulletin board at http://dev.cleversafe.org/forums/, complete with OpenID support. We've also created a development-oriented mailling list, and we likewise encourage you to subscribe to it at http://lists.cleversafe.org/listinfo/dsnet-development.

The first phase of our effort is focused on deploying the right online tools and resources to support you, as a dispersed storage user or developer. We're intensely interested in your thoughts and opinions as to what we should use, so please use the aforementioned means to tell us of your every wish and desire. We'll do our very best to please. Our second phase of work will target the augmentation of documentation and improvement of software usability. We'll be blogging more about this in the very near-term, as we pull everything together, so please stay tuned.

In the meantime, we need your help to grow. We believe this technology can be used for great good in and around the open source software community. Tell us where you think can we can help!

All my best,
John Quigley

With the completion of the 1.0.1 release, we have pretty good code foundation on which to add additional features.  We also have a 1.1.0 open source release coming up which will address a number of issues – particularly performance on globally dispersed (i.e. high latency) networks.

As we continue add new features, I wanted to open up the conversation for suggestions.  So, let us know if you have any ideas or suggestions for new capabilities to add to the Dispersed Storage software.  We’re currently in the midst of a lot of future release planning, so the timing is good for input.

Chris