Trends in data storage and management

This section is titled ‘New directions: students and technology’, but I’m afraid that from the titles, the talks don’t bear out this promise.

However, the first one, on Trends in data storage by Richard Barga from Microsoft, should be interesting, as large portable storage is one of the really fast moving aspects of technology at the moment.

Although the presentation turned out to have no comment on students and technology at all, the issues being tackled here are really of topical interest, as data storage is tied into power usage – and thus global warming. Some of the figures Richard quotes are shocking – e.g. up to 50% of power in a data centre is used to convert current from a/c to d/c and back again (either side of a UPS).

This first thing to say is that our ability to store is growing incredibly (outpacing Moore’s law), and demand for storage is also growing incredibly.

What’s driving storage demand?
Google, Yahoo, etc – all collecting massive amounts of user generated content – who clicked on what etc.
Scientific community – huge datasets

Although increase in disk density is slowing, it is still growing. However, alongside rotating magnetic storage is flash storage. You can now get 16Gb flash drives. This is being driven from the ground up – use in phones, camera,

By 2012 a 1 or 2 Terabyte disk for $400 – not only cheap, but also low energy use (he has already mentioned the fact that if we keep growing traditional storage at current rates, we won’t be able to power it!)

Storage is now cheap (1 terabyte about 1000 dollars). However, the cost of maintaining storage is very expensive (can be $100k per Tb per year).

So – need to look at tiered storage – use cheaper drives to store less important data (for example, may be storing email on expensive storage, where documents in email are often multiple copies – if this can be moved to cheaper storage, you’ve got a saving)

By introducing appropriate technologies you can make significant savings.

Microsoft experimented on the reliability of cheap disks – and found that cheap sata disks are much more reliable than the literature suggests. So, MS, Google, etc. are using large arrays of cheap disks rather than spending twice as much on SCSI drives. Where there are errors, with proper backups and multiple disks, you can simply replaced failed disks, still cheaper.

Now Roger is moving on to some issues close to my heart – the cost of power and cooling in data centers with large amount of storage. Gartner have said that Power and cooling will be a top 3 issue for CIOs in the near future.

It can take as much power to cool a system as to power the system in the first place. Examples of people installing SANs and finding they cannot get enough power to run it (that is, the grid can’t supply them with sufficient power).

One thing that can be done is to use Storage virtualisation – but we are in very early days. But in the above example, they managed to acheive a server consolidation of 30:1 (although 20:1 is more typical)

Coming back to Flash – example of being able to drop power from 500W to 1W by using Flash in place of spinning media.

Racks can’t cool more than 10Kw per rack using forced air cooling alone – water is 3500 times more effective than air by volume – you can also get water-cooled racks (e.g. SprayCool)

So – performance and power demands will continue to grow.
In this context data center thermal capacity is the challenge.
No ‘magic bullet’ – but new Service Oriented Architectures will emerge, and become accepted (e.g. Amazon S3).

With Climate change being a huge global issue, data centres are going to come under increasing scrutiny. Storage management staff skills will need to evolve, educational programmes need to emerge…

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.