In desperate need of Storage Consolidation - Custom NAS?

J

JargonGR

Limp Gawd
Joined
Dec 16, 2006
Messages
503
I am in the process of putting together a new main PC and consequently I am trying to re-arrange everything including storage for my new set-up.

Currently my main PC has the following drives:

5X 1TB WD Green (old)
2X 1TB Samsung F1 (old)
1X 1.5TB WD Green
1X 2TB WD Green
2X 4TB WD RED (not pro)
1X Samsung 840 Pro (37TB Lifetime writes , 75% Health from HD Sentinel Pro)


Hard Disks Not in the machine

3X 1TB WD Greens
3X 1TB Samsung F1 (need to be checked)


Most of the space is consumed by media (HD movies) but there is a critical chunk of space that I must never lose.

Here is the DATA breakdown:

2TB of music that is mirrored in 2X 1TB HDs that are stored elsewhere and they were only used one as I filled them with data. This way if one of their brothers fail in the system I swap one out.

1TB of Family media (photos and some videos)
1TB of Documents and other work related stuff
4TB of material related to my work (critical) and this is backed up every week to an external 4TB HD (WD REDs)

On top of this I have 5 X additional PCs that I need to backup and cater for their storage (wife and 2X young kids).
I also have an LSI 9211 8i controller.
_______________

I want to start with some consolidation and after I finish my build I want to turn my old PC into a NAS and then tune it with newer parts (Mobo, CPU, RAM). This PC is a not so efficient Q9550, 8GB RAM, Samsung 840 Pro.

At the moment since I am still spending for my main PC I can only get a new big Hard Disk so that I create a mirror of all the critical data and then use it in my future NAS build.

I was thinking of going with a 10TB drive but I am torn between the following:

HGST Ultrastar He10 - which one SATA or SAS?
WD Gold 10TB
WD Red Pro TB

Which is more reliable and the one to go with in the future too? Yet, should I go the SAS route and is my LSI 9211 8i sufficient at the moment (will upgrade controller in the future). I know it all depends on the way I will set up my NAS but I am kinda lost. I want maximum data integrity and reliability for my critical data.

A) What drive should I get now

B) Help me plan my NAS starting with my current PC (will turn to NAS) and drives that I can re-format, attach to a RAID array and then start upgrading.
 
Eh, personally I would just retire the old parts and build new or buy new off the shelf dedicated NAS box. I could see turning the old system into a NAS to be upgraded over time for your family data (which is about 4TB right?) and then going custom build or retail nas box for your critical business data.

I'll let others weigh in on 10tb drives.
 
Do you recommend this for power efficiency reasons, effort needed, reliability, performance? I mean the old PC I have it already and since I can't buy a new NAS right now (at least a good one) it can serve as an external backup solution for the whole family.

In the future I was thinking of building my own with a Supermicro Board that comes with a XEON, 6 Sata Ports + 10GBe. Like this:

https://www.amazon.de/dp/B010W32T62/_encoding=UTF8?coliid=I2BJGELI8QRRJ0&colid=2W0NIZCVMBJO6&psc=0

In any case I need a large drive now since I am really worried and want to move all my data in a new home while also keeping backups on the 2X 4TB WD REDs I have. Then I could use all of the other drives for a RAID array.
 
As an Amazon Associate, HardForum may earn from qualifying purchases.
If your doing anything past 4 drives, then a custom box makes some sense.

If what your storing can fit inside of 4 drives (RAID10 or RAID5), then it's really hard to beat the cost, ease of use, and convenience of an off the shelf NAS box. QNAP, Synology, etc.
 
Well given current HD sizes I can certainly fit my data in 4 drives and some of those off the shelf boxes support expansion boxes as well. But then costs begin to rise...

I have a digital library filled with graphics assets used in our design projects. Those range from small files (1-2MB) to 2GB+ files. This library is growing daily and when we add After Effects video to our projects (in the future) then things will change really fast towards faster and more storage needs. That is why I was thinking of the custom route.

Nonetheless, a commercial NAS box would not go to waste but I want fast access (10GBe then), good read & performance and reliability (no issue with those).

Regarding the HD that I need to buy now which of the 10TB models above would you recommned?

Is a SAS drive worth it / needed or is SATA a good option as well.
 
If Backblaze and Google are anything to model after, they don't spend extra for SAS, or even for enterprise level drives. They do heavy redundancy with the cheapest off-the-shelf commodity drives they can find. And they chuck them the instant one even thinks about throwing a SMART error.
 
Ok that's certainly a good point. So I am going with the SATA model and thus I can put it straight to my new PC without adding a SAS controller. Will leave the LSI on the old PC then (was planning on upgrading the LSI too).

Yet, I would be able to sell the SATA drives more easily every 2-3 years compared to SAS drives. I have a lot of friends in my local audio/video hobby community that could use them just for storing movies.

But price wise out of all the drives I listed above the HGST 10TB He is the cheapest and the WD Red Pro the most expensive. (I need to buy from Amazon.de since I have an unused gift card of 400 Euros).


HGST 10TB He SAS - 327 Euros

HGST 10TB He SATA - 336 Euros

WD Gold 10TB - 385 Euros

WD Red Pro 10TB - 402 Euros


There is something strange going on though with this pricing since one review ( 1-star) states that he checked his drive's serial number and there was no warranty on his HGST from Amazon.
 
Last edited:
What motherboard is your computer using? How many SATA ports?

What case are you using?

If you are going to put *EVERY* drive into service you need 18 SATA ports. If you have 10 SATA ports on your motherboard you can use it and the 9211-8i to accomplish that. Alternatively you can get a SAS expander to connect up to 24 drives to the 9211-8i.

I personally use FreeNAS (virtualized) with a M1015 + HP SAS expander passthrough. My server is in a Lian-Li PC-201B case w/2x 5-to-3 bay converters for 22x 3.5" bays. FreeNAS can be booted from USB keys which saves you a SATA port.

If your case is too small you can cheat with this:

https://www.aliexpress.com/item/NEW...c0af-49e2-8e61-db7101f65eeb&priceBeautifyAB=0

HTB130IUbnvI8KJjSspjq6AgjXXa5.jpg


It is 10 bays for only $50 US and free shipping, includes the fans.

Do you have any experience with Linux and ZFS? You need to understand what you are getting into with FreeNAS as expansion can be complicated if you set it up wrong.
 
The PC that I am thinking of using as a NAS is the following:

Motherboard: P5Q3 Deluxe / WiFi-AP - 8 SATA Ports
CPU : Q9550 @3.4GHz - Watercooled
RAM: 8GB (tried to add 8GB more for 16GB but did not work 2 times - I need a 16GB kit probably)
VGA: Nvdia GTX780 (will add a lowly NVIDIA or ATI that I have around)

CASE : Mountain MODs U2 UFO (Original Black) - I won't use it for my new build.

The Case can hold even up to 25 HDs using HD Brackets placed on 120mm Fans - I have extra brackets.
It already has 14 HDs in it. I don't have to install all those HDs though maybe the best 10-14 out of the pack so using the LSI and the M/B ports I am fine.

Now the hard part would be linux & ZFS - no I don't have any experience but I learn fast. I don't think it helps but I do know CSS, HTML, some Javascript (on going learning) and basic PHP stuff. My only experience with Linux is basic web server stuff but I still have some way to go before I am confident in setting up my own web server from scratch.

Anyway, I need to learn this stuff so if I don't get in to it and start studying I will never do. LOL I remember the first time I tried to play with CSS (which is easy) what a frustrating experience it was. But I always insist, read, ask, read some more, rinse and repeat until I've learned.
 
I've had alot of luck with RockStor BTRFS NAS OS. Ran it on an Old i7-870, 16gb of ram, and a pile of old disks.
 
Thanks for the suggestions! Keep them coming.

Regarding my pending order of a single 10TB drive (from Amazon due to gift card) I am narrowing it down to

A) WD Gold
B) WD Red Pro
C) WD Red Plain

The HGST are out since the ones Amazon.de is selling are OEM and don't have warranty. I need to buy from Amazo (not other seller) since this way I am exempted from 19% VAT tax when using our EU Registered VAT number.

So it comes down to those 3 above and the I am leaning towards the Gold and the Pro. My reasoning is better warranty and it will be used in my main PC as a storage device and will be working daily. The data on it will be mirrored / backed up on the NAS. The NAS will also have 2X drives in caddies that turn ON/OFF for offline backups.
 
The chief concern about using your current CPU/motherboard is you are limited to 8GB RAM.

FreeNAS eats RAM like candy, the general rule is 1GB RAM per 1TB storage..

FreeNAS is well-documented so I advise you to read up about it and ZFS before diving in. Knowing ZFS is vital because changing arrays is non-trivial if you do the layout wrong.
 
Hmm I just noticed that the MAX RAM it supports is 16GB but good luck finding a kit that works (tried 3 times and not working). Yet, price wise it might not be worth it.
Maybe I should sell it / dump it and even buy something newer! Even used would be better. Will look into this but I am not spending any money into such an old platform for RAM that is not going to work.

I will read on FreeNAS but I am also considering Unraid.
 
There is a lot of information around regarding ZFS and RAM that is wrong or need interpretations.

Oracle Solaris for example where ZFS is origin and native claims 2GB RAM as minimum and this is enough for a stable 64bit OS with ZFS, does not matter a poolsize. BSD based solutions may need a little more RAM because the internal RAM organisation of ZFS is based on Solaris but that should not be a relevant size.

But stable does not mean fast. ZFS is a CopyOnWrite filesystem. This gives crash resistency and snaps but as every data modification means a rewrite of a whole datablock with checksums and as data is spread evenly over the whole ZFS Raid array even sequential workloads are not as fast as on older filesystems and quite limited by its iops/s performance. You do not get the extra data security without a price. With very low RAM, say 1-3 GB RAM ZFS can be very slow unless you compensate this with fast SSDs datadisks.

ZFS comes with a superior rambased writecache (up to 5s on Solaris, up to 10% of RAM/max 4GB on Open-ZFS per default) and a rambased readcache ARC (most of otherwise free and unused RAM) . If a workload can be handled by these caches your performance is pure RAM performance even with slow disks. The more RAM you have the higher is the propability that you are not limited by disk performance. I have made tests where SMB read write performance via SMB can go > 2GB/s on Oracle Solaris, the fastest ZFS server (you need 40G nics for that).

So ZFS does not need RAM. It is you if you want performance. There is only one aspect of ZFS that is online deduplication. If you want to enable without a huge possible performance degration you should add RAM of around 1-4 GB data per deduplicated pooldata (not poolsize)

For an average 64bit ZFS homeserver I would say 4-8 GB is ok. For a multiuser server with many small files 64-128G RAM can be a proper value. All depends on use cases.

see my ZFS performance tests with different RAM sizes
http://napp-it.org/doc/downloads/optane_slog_pool_performane.pdf
 
Direct Compute Storage spaces and upgrade Lol....I do kinda want one of these mean HDDs now and stupid sizes.
 
I know I need to read a lot on the subject but to begin I would love a motherboard like this.

Supermicro X10SDV-TLN4F

X10SDV-8C-TLN4F_spec.jpg



Besides working as a base for a NAS this can also run some APPs and serve media as well (incl transcoding).

Alternatively as Ranulfo said here:

Ranulfo said:
Eh, personally I would just retire the old parts and build new or buy new off the shelf dedicated NAS box. I could see turning the old system into a NAS to be upgraded over time for your family data (which is about 4TB right?) and then going custom build or retail nas box for your critical business data.

I'll let others weigh in on 10tb drives.
Click to expand...

Maybe I will use the old computer as a backup server with the LSI controller only for growing family media and keep making copies of everything critical (offline backups) on several hard disks until I am ready for either a top of the line Synology or a proper NAS build.
 
Well, I went ahead and ordered a WD Gold 10Tb to use as a the main storage drive on my new PC while planning my storage strategy. I will copy all my business data on it and also keep duplicates on the other drives since I am not feeling secure with my current setup (aging drives) right now plus I am almost out of space unless I start deleting media (BD-Rips) which I might do.

The HGST drives from AMAZON.DE had no warranty, the Red Pros were not available by them and wanted something faster than plain Reds for this use case.
 
As an Amazon Associate, HardForum may earn from qualifying purchases.
I use a X9SCL with a G2020 (ivy bridge) with 12 GB ECC RAM. Plenty powerful if you aren't using VMs and just sharing files. The board has two Intel NICs for LACP. You can get the board/chip on ebay for around $60 total.
 
You must log in or register to reply here.
Back
Top