Need recommendations for a workstation build

[bash64]

reposting from another section:

Hello,

I am on my 3rd MSI gaming laptop and laptops just won't cut it for me anymore.
I am into long and heavy video processing using h.264.
I also do large zip file builds.
I'd also like to get into rendering.

I'd like recommendations on a ryzen build and amd video cards.

I need at least two video cards (non crossfire), three video cards would be ideal.
16 cpus minimum (8 cores, 2 threads per core is fine).
32gb of ram or more...
9 drives (2 for raid mirror, 5 for raid 5, 2 for hot spares).
Gaming is only a secondary issue. I assume it will game.
I do NOT overclock. Ever. Turbo boost is ok if its a factory option.
Must run cool for long periods of time under a heavy load.
I hear that ryzen can run hot at 85c, but somehow this is not harmful.
My experience with Intel and heat has not been good. Gets way too hot under a heavy load.

AES support in the cpu is mandatory.
IOMMU (vt-d, amd virtualization, whatever they call it) is mandatory.

Looking to spend up to $3000 (without monitors).
Not doing 4k. 1080p is fine. Already have one 27" 1080p monitor.

I use Linux exclusively, though I can run windows using virtual machine software (qemu w/gpu passthrough or vmware).

Questions:
What about this 3200mhz (2 sticks) vs 2400mhz (4 sticks) ram for ryzen? I prefer speed.
What about amd firepro cards for rendering? I hear openCL is not as good as CUDA. Should I go nvidia instead?

Thanks in advance for any suggestions.
And, yes, you can tell me Intel is better if you want.

 

[Bandalo]

1. Move to a NAS. That's too many drives to be sticking into a desktop for long periods of time IMO. Run your mirror locally, and stick the other 7 drives in a NAS enclosure on a nice UPS.

2. AMD is more limited on PCIe lanes than the Intel HEDT or Xeon chips. So you CAN run multiple GPUs, but you may be limited down to PCIe 4x for those cards. The best you can do with Ryzen is 8x + 8x.

3. Get a good AIO cooler, and you'll keep it ~60C at full load forever. A really good air cooler will be 10-15C hotter. This is true to for both Intel and AMD, getting hot under load is not an Intel-specific issue.

4. Faster memory is probably better with Ryzen. 2 vs 4 sticks shouldn't matter, since you're dual-channel only. Just make sure you check the QVL for the motherboard you buy. Ryzen is a bit more picky about RAM than Intel chips.

 

[bash64]

Thanks for the response:
I am concerned about slowness when placing drives in an external enclosure.
I have seen speeds of 300mbs/sec internally and esata/usb 3 speeds of only 120mbs tops.

I was unaware of AMD having fewer PCIe lanes. That could be an issue.

 

[Bandalo]

Thanks for the response:
I am concerned about slowness when placing drives in an external enclosure.
I have seen speeds of 300mbs/sec internally and esata/usb 3 speeds of only 120mbs tops.

I was unaware of AMD having fewer PCIe lanes. That could be an issue.

I'd personally put those drives in an external NAS and run 1Gb Ethernet. If the speed is a requirement over just a "nice-to-have", then an external Thunderbolt enclosure might work. Understand that will ALSO consume PCIe lanes. Speed has a cost, no matter which route you take. I'd just hate to put all my eggs in one basket and have that much hardware in one box.

The issue with PCIe lanes may or may not make a difference. Running most graphics cards a 8x isn't limiting, especially at 1080p. If you're using the cards for rendering though, that may be different. I honestly don't have any information on how rendering is impacted by running on a narrower PCIe bus. Might be insignificant, it might be a huge deal.

 

[bash64]

I am concerned about the NAS limiting what filesystems I use.
Currently, I run Linux and use softraid (mdadm) and LUKS encryption, and LVM (logical volume manager).
Admittedly, the raid 5 drives and the two hot spares is for off loading large files so technically it is external storage.

 

[Bandalo]

I am concerned about the NAS limiting what filesystems I use.
Currently, I run Linux and use softraid (mdadm) and LUKS encryption, and LVM (logical volume manager).
Admittedly, the raid 5 drives and the two hot spares is for off loading large files so technically it is external storage.

Well, if you're using a separate NAS, your local machine will just see it as network shares. Specifically what RAID levels, encryption and file systems you use on the NAS will depend on what OS you run on the NAS itself. I

 

[Nobu]

1. Move to a NAS. That's too many drives to be sticking into a desktop for long periods of time IMO. Run your mirror locally, and stick the other 7 drives in a NAS enclosure on a nice UPS.

2. AMD is more limited on PCIe lanes than the Intel HEDT or Xeon chips. So you CAN run multiple GPUs, but you may be limited down to PCIe 4x for those cards. The best you can do with Ryzen is 8x + 8x.

3. Get a good AIO cooler, and you'll keep it ~60C at full load forever. A really good air cooler will be 10-15C hotter. This is true to for both Intel and AMD, getting hot under load is not an Intel-specific issue.

4. Faster memory is probably better with Ryzen. 2 vs 4 sticks shouldn't matter, since you're dual-channel only. Just make sure you check the QVL for the motherboard you buy. Ryzen is a bit more picky about RAM than Intel chips.

+1 on PCIe lanes, NAS, and memory, with a couple caveats:
Lane limitations will heavily depend on the motherboard and what chipset it uses: I'd recommend thoroghly researching motherboard options before buying, if you research nothing else.
The speed your RAM will reach depends on a number of factors (maker: hynix, samsung, etc; ranks: 1r, 2r...; number of modules: 2, 4...). Generally, Samsung B-die has been found to be fastest, but there are others that perform well too. I believe AMD said 2, single rank, single-sided modules will perform best, but ymmv.
For the NAS, you could use two ethernet cables to effectively double the bandwidth, but a board with two ports may limit your PCIe lanes. A network card on a board with just one port may sidestep the issue, as long as it doesn't use, block, or disable a slot you had planned on using. Again, motherboard research is important here.

 

[Xyvotha]

- +1 on NAS, at least for the RAID 5 part.
- You said up to $3K, does that include video cards? I think that limits how much you can spend in MB/CPU/RAM.
- Your video processing applications rely more on CPU or GPU? If it's the former go for R7 1800X (you said you won't overclock, else I'd say 1700). If they're GPU intensive go for more PCIe lanes on X99 and fewer cores(still thinking about that $3K ceiling).
- Big zip files: depending on the application, a dedicated cache/page SSD could help a lot.
- You didn't mention noise requirements, only temps. Good air cooler like the Ninja 4 will keep a stock CPU in decent temps and save you some $$ for the rest of components.
- Ryzen memory support is being improved, but for now I'd go for 2 sticks instead of 4 in a production PC.

Cheers.