Your data is going to be stored on sheets of glass soon

(image: William Warby)

Almost all of us who work in museums will have experienced some degree of struggle with the sheer amount of data we are having to manage these days.  Whether it’s the act of scanning art works at mind-bogglingly high-resolution rates for archive, or storing and transferring video footage at 4K and higher resolutions – we’ve all been tackling RAID arrays, ditching the JBOD approach (Just a Bunch of Disks), and eyeing up cloud-based storage solutions.

So, it will be pleasing to hear that Microsoft have been building on technology developed by the University of Southampton to produce a high-capacity storage medium made of quartz glass.

In a wonderfully nostalgic touch, the first major demonstration of this technology was the act of storing the entirety of the 1978 Warner Bros. classic movie Superman on a piece of glass around the size of a beer mat, and only 2 millimetres thick.

Those with keen memories will recall that in the film Superman, the eponymous hero would retreat to the sanctuary of the Fortress of Solitude, a gigantic crystalline structure, that held within it ‘memory crystals’ – transparent shards that encapsulated data used to orientate the young superhero.

In more Earthly concerns, we are apparently currently entering the Zettabyte Era (that’s 1000⁷ bytes, or a trillion Gb to you).  We seem to have completely skipped the Petabyte and Exabyte ‘eras’, but perhaps they don’t have zingy enough names! 

In any case – we’re all going to need a lot more data – and that is the storage challenge that this technology is being developed to address.

It works by firing an ultrafast laser into the structure of the glass.  This produces a permanent 3-dimentional nanostructure called a ‘voxel’.  The beauty of the voxel is that it basically remains stable for the lifetime of the piece of glass itself, which is to say – potentially many hundreds of years.  This longevity eclipses even the most robust storage techniques currently on offer. 

Data access is also a fast process.  Richard Black, Microsoft principal research software engineer has stated –

If you’re old enough to remember rewinding and forwarding songs on cassette tapes, it can take a while to get to the part you want.  By contrast, it’s very rapid to read back from glass because you can move simultaneously within the x or y or z axis.

However, accessing data and correctly interpreting it are two different matters, and the latter presents some unique challenges.  In a densely populated optical medium like glass it can be hard to visually distinguish one distinct object from another. 

Each voxel has its own unique shape, but they are each surrounded by hundreds of other nearby voxels.  Visually sorting shapes is something that computer algorithms find relatively tough (humans are pretty good at it – that’s why we use Captcha images to keep out bots!).

To navigate this problem the team behind Project Silica have employed deep-learning algorithms – AI structures that keep on developing strategies for best identifying voxel shapes and extrapolating data from them.  This is a solution that is becoming very popular in tackling signal processing challenges including voice-recognition, video analysis, etc.

This storage technology is aimed initially at cloud-based facilities, but it’s not unlikely that it will make its way into consumer data storage solutions at some point in the future.  Either way, it is likely to be a boon to museums aiming to archive their own digital collections.

As a passing note – this is not the first time that glass has been used as a storage medium.  Early records were sometimes cut on glass, either as master copies or for novelty value.  Prior to this project, archivists at Warner Bros. had discovered some unique glass records that had been stored away in their collection since the 1940’s. 

Brad Collar, an archive executive at the corporation has stated:

We actually found players that we could play these things back on, and they were just as good because they were stored on glass. And we were able to digitize and save those wonderful pieces of content.

And that content?  Coincidentally, it turned out to be 1940’s radio serialisations of… Superman!

So now one of our oldest assets in our vault is glass and one of the newest technologies in our vault is glass – and they’re both Superman. So, we really have come full circle.

He may be the ‘Man of Steel’, but he appears to be a good candidate for being the Mascot of Glass as well (Captain of Crystal? Queen of Quartz? Your entries below!!!)

Now, where did I put that removable drive..?