this may be of interest, if a little redundant: http://www.straightdope.com/columns/020816.html

oxlongm wrote:I saw Amazing Translucent Sealing Shit back in '93 on the second stage at H.O.R.D.E. The solo on Unbreakable Plastic Slab that night was freaking transcendent, man.

yeah, maybe... but the earlier stuff was better. before they went all 'guitar'

zeroart wrote:
if you are saving files in wave or aiff, you are covered for however long digital media can hold up. there is an open C library that reads/writes lots of different file formats and can handle both little-endian and big-endian data. it is called libsndfile.

i'm not sure how the argument of the longevity of digital media even comes into play. it is a magnetic medium just like analogue tape. by all rights a magnetic hard disk should last just as long as analogue tape. we don't know exactly how long, but there are magnetic hard drives that contain data still intact from the late 1950's.

I fully agree with you here.

I know we shouldn't ramble on with this old topic, but I don't see where all the worry has come from.

Here, with respect to the digital algorithm, we are not dealing with any compressed formats. We are dealing with a standardised digital represention which is unlikely to change greatly. It is simply a list of sample values in respective order. File conversion is not going to be a problem. It's like MIDI. It is a standardised protocol. It can easily be converted if necessary and can be recorded onto a medium for safe storage.

If there is a demand for something (say a simple file converter) then there will be a market for it. I don't believe that .wav or .aiff will ever be completely unsupported.

With respect to hardware, again, if people want to safely store digital audio, they can do this. If a piece of audio is important, then back it up. Until recently, I had all of my music from my Amiga computer which was twelve years old and the formats still fully supported.

zeroart wrote:i'm not sure how the argument of the longevity of digital media even comes into play. it is a magnetic medium just like analogue tape. by all rights a magnetic hard disk should last just as long as analogue tape. we don't know exactly how long, but there are magnetic hard drives that contain data still intact from the late 1950's.

The data density of hard disk drives from the 60s (it would have been core memory and paper tape or punch cards in the 50s) was much lower than it is now, so conventional oxide-based media were used. These will last a long time. Current (since about 1980) magnetic data storage is orders of magnitude more dense, is recorded on metal particle- (as opposed to metal oxide) media, and much of it is helical-scan tape. Helical scanning makes the physical flatness of the tape critical, as the data are recorded in diagonal stripes meant to be read sequentially, rather than a linear stream of data. Metal particle tape is unstable and changes physically when it oxidizes. Oxide coatings are already oxidized, and don't change very much over time. Most of that change is in the binder and plastic base, and those changes are not catastrophic. Metal particle media are also subject to this substrate deformation.

These things have compounding effects on data storage: Helical scanning means any small defect will lose data. High density means that any loss will be a lot of data lost. Metal particle coatings and plastic substrates ensure that there will be defects over time.

The only way to preserve digital data is to continuously copy files onto ever newer media and formats, and to translate them so they can be opened by the software of the day. Nobody is doing this.

I work in a laboratory that has to handle massive amounts of data which is updated daily. We currently use two digital backup systems in rotation, but when it gets down to the archives, we still write to tape.

I think the problem of standardization is a big one for more than just music, particularly as the western world is in the process of digitizing all of its information systems. I'm sure digital archiving standards are in the works, but I don't really know anything about it.

Incidentally, digital technology makes it very easy to transfer information onto the 'working format' of the times, at minimal loss of quality (we're talking on the order single data points), and the process can be automated.

But I'm still skeptical about putting my faith in the guts of such a complex machine (not to mention letting it automate the backup process).

Even my little akai digital demo machine scares the bejeezus out of me every time I turn it on. I'm still waiting in fear for the word 'corrupt' to pop up onscreen. terrified. Every time I turn it on.

steve wrote:

zeroart wrote:i'm not sure how the argument of the longevity of digital media even comes into play. it is a magnetic medium just like analogue tape. by all rights a magnetic hard disk should last just as long as analogue tape. we don't know exactly how long, but there are magnetic hard drives that contain data still intact from the late 1950's.

The data density of hard disk drives from the 60s (it would have been core memory and paper tape or punch cards in the 50s) was much lower than it is now, so conventional oxide-based media were used. These will last a long time. Current (since about 1980) magnetic data storage is orders of magnitude more dense, is recorded on metal particle- (as opposed to metal oxide) media, and much of it is helical-scan tape. Helical scanning makes the physical flatness of the tape critical, as the data are recorded in diagonal stripes meant to be read sequentially, rather than a linear stream of data. Metal particle tape is unstable and changes physically when it oxidizes. Oxide coatings are already oxidized, and don't change very much over time. Most of that change is in the binder and plastic base, and those changes are not catastrophic. Metal particle media are also subject to this substrate deformation.

These things have compounding effects on data storage: Helical scanning means any small defect will lose data. High density means that any loss will be a lot of data lost. Metal particle coatings and plastic substrates ensure that there will be defects over time.

The only way to preserve digital data is to continuously copy files onto ever newer media and formats, and to translate them so they can be opened by the software of the day. Nobody is doing this.

while a bulk of this information is true, it's all solvable. much of the information gathered and stored at IBM during the mid to late 50's was stored to the magnetic hard disks you describe.

my friends at XEPA digital are doing exactly what you say in your last paragraph, steve. also, iron mountain does the exact same thing for all other digital information by providing media storage and data protection. they come by my place of employment daily to take the drives of data from the nightly backups of my server farm.

this debate could continue forever. there is no solid answer. it would seem to me that if you had a master recording on a medium and in a format that you wanted to keep, you would hold onto or keep in possession a device to get to your data. you don't record an album to tape and then get rid of your tape machine.

gio makes some fine points as well. like i said before, there is quite a way to go yet, but it is definitely all solvable. i actually can't believe this has stayed civil. kudos on a community of people that bring backed up facts.

zeroart wrote:also, iron mountain does the exact same thing for all other digital information by providing media storage and data protection. they come by my place of employment daily to take the drives of data from the nightly backups of my server farm.

All we do is store your drives. Unless you are digitally archiving your stuff (but reading your post it doesn't sound like it) so that means all IM is doing is taking your stuff offsite. One of the things I understood of Steve's post was that no one is continually making new copies of the data on surviving formats. Your drives will be sitting in our warehouse and stay safe (most of the time, I drop stuff sometimes) but it's not being updated on new formats.

zeroart wrote:while a bulk of this information is true, it's all solvable.

To say that it is "solvable" is not to say that it has been solved, or that there is anything in place now that alleviates my concerns. The problems have been "solvable" for 30 years, and still no one implements the solutions. They don't because it takes time and money and is a pain in the ass. Putting a tape on the shelf takes no effort at all.

my friends at XEPA digital are doing exactly what you say in your last paragraph, steve. also, iron mountain does the exact same thing for all other digital information by providing media storage and data protection. they come by my place of employment daily to take the drives of data from the nightly backups of my server farm.

And when your company or Iron Mountain goes broke, as happens to all companies, and the bill doesn't get paid for this continuous re-copying of data, what happens then? Poof? And if I hire a company to do it for me, do I now bill all of my clients every month for eternity for the cost of keeping their session masters up to date?

this debate could continue forever. there is no solid answer. it would seem to me that if you had a master recording on a medium and in a format that you wanted to keep, you would hold onto or keep in possession a device to get to your data. you don't record an album to tape and then get rid of your tape machine.

No, I record an album and give the master to the band and they disappear into the wind. I owe it to them to give them something they will always be able to play, not something they must continuously repair in case it disappears forever.

I apologize if this has been covered elsewhere, but could somebody post a link to a well-vetted or at least thoroughly trustworthy study that shows that modern hard disks are expected to not retain data flawlessly if left in storage for decades? I've heard that the disks must be intermittently refreshed or they'll lose at least some of the data (something to do with retaining the proper magnetic charges) but have never seen a source cited for that one. I'm not disputing the validity of the notion, just curious to see a good paper explaining it.

Gone Savage wrote:

zeroart wrote:also, iron mountain does the exact same thing for all other digital information by providing media storage and data protection. they come by my place of employment daily to take the drives of data from the nightly backups of my server farm.

All we do is store your drives. Unless you are digitally archiving your stuff (but reading your post it doesn't sound like it) so that means all IM is doing is taking your stuff offsite. One of the things I understood of Steve's post was that no one is continually making new copies of the data on surviving formats. Your drives will be sitting in our warehouse and stay safe (most of the time, I drop stuff sometimes) but it's not being updated on new formats.

only certain backups on the rotation are digitally archived from my day job. those too are stored with you guys.

i'm pretty sure that XEPA digital has an agreement with you guys to digitally store and access the information they archive in the deep archives.