With the completion of the 1.0.1 release, we have pretty good code foundation on which to add additional features.  We also have a 1.1.0 open source release coming up which will address a number of issues – particularly performance on globally dispersed (i.e. high latency) networks.

As we continue add new features, I wanted to open up the conversation for suggestions.  So, let us know if you have any ideas or suggestions for new capabilities to add to the Dispersed Storage software.  We’re currently in the midst of a lot of future release planning, so the timing is good for input.

Chris

Now that Cleversafe has announced its upcoming commercial products, we can talk more clearly about the relationship between the Dispersed Storage open source project (here at www.cleversafe.org) and Cleversafe’s commercial products. 

As we are doing with our technical approach, we are utilizing the outstanding and proven approach taken by the Internet for our business ecosystem model. 

The key protocols that power the Internet:  TCP, IP, UDP, LDAP, DNS, etc are genuinely open protocols in that they are in the public domain and available as open source.  As a result, the Internet really is an interoperable internetwork.  Way back in the 80’s, I was in the IT department of a large aerospace company responsible for setting networking standards and I remember when other networking protocols – such as IBM’s SNA, DECNET and Novell’s SPX/IPX – were more popular than TCP/IP.  Each of these proprietary protocols were driven by well-funded and capable R&D organizations; whereas, the R&D behind TCP/IP was not fueled by a large, well-funded technology company.  However 20 years later, TCP/IP has pretty much completely taken over as the networking and internetworking protocol of choice and the Internet absolutely dominates any networks still using these previously popular proprietary protocols (P4’s).

Yes, TCP/IP is a great protocol, but the P4’s protocols were pretty great, too.  I believe part of the reason that TCP/IP and the Internet emerged as the world’s network is based on human nature.  Simply put, most people didn’t want the world’s network to be owned or controlled by one company, so as the world built its internetwork, a genuinely open protocol had an inherent advantage and was ultimately propelled to become the protocol behind the Internet, the world’s network.

When we began to develop Dispersed Storage, we realized the technology had significant potential, namely to store the world’s data.  We also realized, based on the lesson of the Internet, that the protocols of a Storage Internet ultimately must also be open source, so we created the Dispersed Storage project  in order to develop and publish the Dispersed Storage protocol as open source.  In order to do this, we also created a lot of new open source code and incorporated a lot of open source code from other projects.

Even though the Dispersed Storage protocol stack and core features are available as open source, plenty of great opportunities exist for creating commercial products and services.  To use the Internet analogy again, network equipment companies, like Cisco and Juniper create commercial products (switches, routers, gateways, etc.) by taking the open TCP/IP protocol, adding proprietary features like management systems, integrating onto optimized hardware with an optimized OS and then selling those products through a trained channel that provides services like support and installation.  Cleversafe is using that same approach for its commercial products, so once again we are standing on the shoulders of the giants who build the technology and the business models for the Internet.

Chris

In preparing a presentation recently on long term trends in data storage, I was talking with Russ Kennedy and he mentioned that it was a known fact within the data storage industry that hard drive performance has been lagging.  So mentioned this to Dennis Roberson and he connected me with this article at Tom’s Hardware which I found very enlightening.

It turns out that increases in hard drive capacities have been keeping pace with Moore’s law (as observed) by doubling every 24 months.  But hard drive performance (reads and writes) hasn’t been keeping pace.  Even though hard drive performance has been increasing, it hasn’t been increasing as fast as hard drive capacities have been increasing. 

So if the amount of data that people want to read from and write to hard drives has been increasing with the rate of increase of CPU performance and hard drive capacities, then the realized performance of hard drives (on this “technology adjusted” basis) has been getting SLOWER!  This is a big part of why Windows never seems to load any faster.

The article at Tom’s Hardware really brought these diverging trends together in a compelling way by measuring the speed to read a single platter against the year when the hard drive was manufactured.  This chart shows that the speed to read a single platter of data has decreased by almost 10X over the past 15 years.  Wow. 

If these trends continue, hard drive speeds will become an increasingly limiting factor which will further limit the approach of a local hard drive as the primary data storage system.  Especially for high performance environments, the solution to lagging drive performance will be architectures like Dispersed Storage that write to or read from multiple drives in parallel.

Chris

“How did you get the initial idea for Cleversafe?” is a question I am asked fairly often possibly since I don’t have a long career background in data storage systems or in coding algorithms. What lead me to that idea of building a geographically distributed Dispersed Storage grids was that I was looking for a way to store my personal data and I had been reading a lot about the history of cryptography.