• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

What solution for data archiving?

3

3d_compos

n00b
Joined
Apr 24, 2011
Messages
22
Hello.

I need a 4 GB external drive for data archiving purposes. The main goal is reliability. The second - data transfer.

What solution will be better?

1.

All-in-one package: WD My Book 4TB USB 3.0 Black

http://www.komputronik.pl/product/203944/Sklep_marki/WD/WD_My_Book_4TB_USB_3_0_Black.html


The drawback is that the actual drives in this product are most likely the WD Green series HDDs. Which are not as reliable as server-class HDDs that WD offer.

2.

To buy separately a "red" class WD HDD and the high quality external enclosure:


WD Red 4TB WD40EFRX (64MB, SATA/600)

http://www.komputronik.pl/product/1.../WD_Red_4TB_WD40EFRX_64MB_cache_SATA-600.html

SilverStone TS07B USB 3.0 - ( SST-TS07B )

http://www.morele.net/kieszen-silve...b-3-0-czarna-sst-ts07b-504425/?ceneo_spo=true

The second solution is 30 percent more expensive. Is it worth considering?
 
Buy more than 1 drive and make 2 copies of your data. Possibly keeping one of the drives off site. Putting all data on a single drive is is a risk even if the drive is kept offline and almost never used.
 
Data archiving=long term storage
In such a case you must fight against silent data errors that occur and can only be detected on access or regular checksum tests

If your archival data is valid
- use checksummed data, best a checksummed filesystem like ZFS
- do regular checks/ scrubs to detect errors
- best is redundancy like a mirror or any other raid level that offers redundancy to repair detected errors

If a data loss or data corrupction is a serious problem
- buy one or better two cheap HP microservers with a ZFS appliance like FreeNAS or Solaris/OmniOS with my free napp-it
- use a redundant setup (mirror or Raid-Z1-3)
- do regular scrubs=checksum tests with data repair

You can remove a mirror/raid archival disk set and put elsewhere.
You should insert from time to time to scrub/repair faults
 
Thank you for your suggestions. They are all valuable.

But can I ask for advice, what option out of two provided, is the best? I'm not considering other solutions right now.

And of course, I plan to buy several identical sets to ensure redundancy.
 
just opinions:

if you go with zfs and familiar with linux
you are lucky, ZFS on linux has stable V0.6.3.

I am migrating Hardware raid6 to ZoL as my main 24/7 homeserver. I do have two backup server.
Honestly, I am still waiting btrfs to support ZFS raidz2/3 alike.


I totaly agree with cold storage is Bluray Disk. atleast 2 disk created, and maintain in the closet. as I remember, disk has life-span (FYI).


if you need a server, TS140 would be a good bet than HP microserver. TS140 more power.
Honestly. my mini-itx celeron baytrail is bit faster than mine n54l :p....
n54l is aging enough. unless you can get cheap enough on n54l, you are good. let says ~$100 including memory 4G minimal


I did backup on DVD for all family photos for 1 years, it is stockpilingin the spindle. and stop to use DVD afterthat, and use two backup server (ZFS on Linux, while waiting btrfs raidz2/3 alike).
photos takes much disk space know, including family videos.
take pictures and videos since my son&daughter born :D.
 
for long term backup:

use a (win)rar like solution that has the ability to create recovery records.

Calculate the expected size of the archive, calculate the expected size of the recovery record using 5-10 percent, then adjust the archive split so that it nearly fills a bluray after already having the recovery record file on the bluray (BD+r size - recovery record size = space available for file archive splits)

Burn dvd/bluray discs with with the recovery record on each disk. check the base archive for errors every year


oh yeah, and for the online/cloud options i would just use glacier.
amazon glacier and the same winrar procedure for blurays.

amazon glacier's benefits are that you can move the archives from glacier to S3 or an EBS solution, and use an EC2 server to process the files you need from the entire archive very fast, then you only download what you need after extracting the files.

using S3/EC2 is great because you only pay for the time that the 'server' is powered on. its a bit complicated to setup but can be blazingly fast for working with backups on amazon's storage system

http://aws.amazon.com/blogs/aws/archive-s3-to-glacier/

keep in mind this is meant for long term with retrieval restrictions and fees (time to retrieve your files due to storage not being on spinning disks in the data center, as well as fee for the data/time the medium is in the server.. gets kinda complicated, but if you have access to fast Inet its not very expensive to retrieve)





for daily access/backup: a nas like the n54l running freenas(gives more control than just an apple time capsule), or a more universal hardware solution, running an ITX board with atom processor, this can also process your rar files for backup to bluray/glacier without having to use your primary workstation. and if you need more processing power, they have ITX boards that can even accept full intel cpu's


example :
http://www.newegg.com/Product/Product.aspx?Item=N82E16811352027
http://www.newegg.com/Product/Product.aspx?Item=N82E16813157475
and any other required parts.
 
Last edited:
You must log in or register to reply here.
Back
Top