is it a dumb way of storing data??

[kenny1999]

Hello I have approximately 50T of data stored on various hard drives, both internal and external. I don't have any online backup solution, the reason for that is I am sick of the slow speed of uploading things. In fact, among 50T of the data, only 20% of them is critical which means the rest 80% of data could be lost without problems, but I don't have a habit of deleting anything even if it's less important, unless it's gone because of defect. Now, my only concern is the 20% of the data I am doing with.

Assume that the cost of external / internal hard drives isn't a problem to me (since I can get the drives almost free because of the nature of my work), can I copy all the critical data from the older drive to a newer drive once or twice a year to reduce the probability of hard drive failure?

Is it a dumb way or one of the recommended method by data genius?

Thank you.

 

[beers]

What's your mechanism currently?

Is it a single array or is it just a bunch of drives?

 

[ssal]

50TB of data! Sound like you had not delete any file since the inception of PC back in the 80's. The way you described it, you probably do not have a good catalog system to tell you what is where. And if you have not used it in a year, or two, you'd probably never gonna use it anyway.

It would be a pain, or impossible, to go thru all the 50T of files to determine what to keep and what not. You may wipe up a short routine to read the last access date/time of the file attribute and move those within the recent years and move them to a 2-3 TB hard drive and start to play a more active role to manage those data. For the rest, you may want to either store it some place and forget about it.

I have about 10-12 years of digital photos (I am a photographer) amounting to about 100 gb. I maintain 3 sources of copies internal and external. I only retain the most recent 5 years actively on my active hard drive. The rest are stored externally. I have a habit of creating a folder for each yymm in my Picture main folder. Between the date and window search, I can find what I am looking for, even it's 10 year old, without much difficulty.

 

[Geoff]

With most hard drives maxing out around 8TB per drive, that's a lot of drives. I would hope this is in a redundant RAID array and not just copied on several individual drives.

Copying data to new drives is a horrible idea, as brand new drives have a not so insignificant failure rate, and you'd have a high chance of losing your data given the number of drives you'd need.

I would look at purchasing a home file server or NAS to store your data, which is running some form of RAID as well as a backup solution for your 20% critical data. This will also make it easier to manage as you'd only have one logical volume and not 10+ individual drives.

I favor Synology, so I'd look at something like this: http://www.amazon.com/Synology-Amer...d_cp_pc_2?ie=UTF8&refRID=19C2JS9AJM5B4Y9G1Z05

 

[beers]

can I copy all the critical data from the older drive to a newer drive once or twice a year to reduce the probability of hard drive failure?

Also to note, if you had a RAID array for these you could simply add new drives as hot spares and then remove the other older drives if needed. The array rebuilds itself within the single contiguous volume and you don't have to manually manage files.