[Build Log] Excelsior! Threadripper KVM Build with GPU Passthrough and ZFS

TheHobbyist

Hugs Hard Johnnies [H]ard
Joined
Apr 8, 2008
Messages
456
Greetings True Believers!

Lets go on a journey together. Join me as I plan and execute an amazing computer for myself and my family that will go by the name of “Excelsior!” in honor of the great Stan Lee.

To live up to its name, it must be something truly exceptional. My vision for Excelsior! is that it will have a host such as Proxmox that supports GPU Passthrough as well as ZFS and it will serve as 2 Windows 10 Gaming Workstations, a NAS, and whatever else we can dream and implement!

The Hardware:

  • Processor: AMD Threadripper 2950X
  • CPU Cooler: Cooler Master Wraith Ripper
  • Motherboard: ASRock X399 Taichi
  • Memory: 128GB DDR4 ECC
    • (8x) Crucial 16GB DDR4-2666 ECC UDIMM
      • Part #: CT16G4WFD8266
      • Crucial Part #: CT12146164
  • Storage:
    • (3x) Samsung 970 Evo 500GB M.2 NVME
    • (8x) Seagate 12TB IronWolf ST12000VN0007
  • Video Cards:
    • GPU (Host)
    • GPU (Windows 10 Guest)
    • GPU (Windows 10 Guest)
  • Case: Fractal Design Define R6 Blackout FD-CA-DEF-R6C-BKO
  • Power Supply: Seasonic PRIME 1300 Platimun SSR-1300PD
Excelsior! PC Part Picker

The Software:

  • Host:
    • Proxmox
    • ?Other Linux KVM?
  • Guest:
    • Windows 10 with GPU passthrough
      • Main Workstation
    • Windows 10 with GPU passthrough
      • NVIDIA GameStream VM to NVIDIA Shield TV in living room
      • Parsec Server
Research, Design, Planning:

  • Phase 1: Research and Design
  • Phase 2: Acquire Hardware
  • Phase 3: Windows 10 Bare Metal
    • Hardware Stage 1:
      • Windows 10 Bare Metal Install
        • OEM Cooler Master Wraith Ripper Thermal Interface Material
        • Single GPU
        • Single NVME
        • No 3.5" HDDs
      • Overclocking and Performance Benchmarking
    • Hardware Stage 2:
      • Windows 10 Bare Metal Install
        • Replace OEM Cooler Master Wraith Ripper Thermal Interface Material with Prolimatech Pk-3 Using [H] Method
      • Overclocking and Performance Benchmarking
  • Phase 4: Linux
    • Hardware Stage 3:
      • Dual Boot Windows 10 Bare Metal and Linux
        • Single GPU
        • (2x) Samsung 970 Evo 500GB M.2 NVME
        • (8x) Seagate 12TB IronWolf SATA HDD
      • Learn Linux, Debian, ProxMox, Arch Linux, ZFS
      • Decide on hypervisor and final hardware configuration
  • Phase 5: Virtualized
    • Hardware Stage 4:
      • Single Boot Linux Hypervisor
        • Full Hardware Build
          • Boot from ZFS Mirrorred USB 3.1 Drives
          • Multi-GPU
          • (3x) Samsung 970 Evo 500GB M.2 NVME
        • Windows 10 Guest 1
          • 8 cores, 16 threads
          • 64GB RAM
          • (1x) Samsung 970 Evo 500GB M.2 NVME
          • Single GPU Passthrough
        • Windows 10 Guest 2
          • 4 cores, 8 threads
          • 8GB RAM
          • (1x) Samsung 970 Evo 500GB M.2 NVME
          • Single GPU Passthrough
      • Performance Benchmarking and Comparison vs Bare Metal
  • Phase 6: ...

The Build:


g9gLcgo.jpg


It all starts with a blank canvas.

fvNIuMq.jpg


Hardware acquired!

aOgNiP6.jpg


Don't forget the case that comes in an absolutely massive (but protective!) box.

AQpAxnm.jpg


Lets invite some old friends to the party.

ijz2pg7.jpg


2 SATA cables. Hmm... that's not gonna cut it!

2aQ0W9y.jpg


A beautiful motherboard, but it looks lonely. I think a team up is in order!

IXyQPlu.jpg


Tai Chi, meet Threadripper. This unboxing was very satisfying, not gonna lie. An included torque wrench? Gigity!

kM1hQrm.jpg


Alright, now they need to figure out how to work together.

nmTQ6W4.jpg


This is my first go at a Threadripper build, lets see if I can NOT BREAK IT.

5Xb2W5b.jpg


Oh lawd... so many fragile pins exposed...

13qzGXY.jpg


It's a fascinating install process. It feels very substantial and well engineered.

8j3Ajw6.jpg


Threadripper in place. Almost there!

ohoUbWn.jpg


Phew. Threadripper safe and sound in its socket.

bH9knBG.jpg


Watch out Tai Chi! Ninjas! Oh, wait, you guys wanna team up too?

HNNJfyd.jpg


Lets see what you've got Ninjas.

0a9P2CU.jpg


Triple flying ninja kick!

lpvKFyg.jpg


Man, I can't believe you guys landed that.

9UcsOvX.jpg


Our team is growing in infamy. Mighty champions flock to us to join our righteous cause.

K9dZxkl.jpg


These emerald warriors fit right in. The time before they were part of the team feels like a distant memory.

Z6mt0Ab.jpg


Watch out! A Wraith Ripper! Be you friend or foe?

mZYCFR0.jpg


Wraith Ripper, with the help of your friend T.I.M., I think you and Threadripper will work well together.

fumQiy4.jpg


What an overwhelming presence!

NPpGDkK.jpg


The protective films are off. This is getting serious! Wraith Ripper, such a towering behemoth!

I will return soon to further flesh out this first post, but for now I eagerly await your feedback, whether it be on hardware choices, system design, software advice, or otherwise!

Excelsior!
 
Last edited:
Johnny, do you play baseball? lol!



LOL perfect, a fellow fenslerfilm fan to boot - if you're interested I have an ISO of that original DVD from before it got lawyered and I don't know of anyone with the full DVD rips uploaded. Can slowly upload it to mega for you if you wish. Just post lots of build porn later lol
 
I had some time to work on the build and added some content to the OP. Enjoy!
 
LOL perfect, a fellow fenslerfilm fan to boot - if you're interested I have an ISO of that original DVD from before it got lawyered and I don't know of anyone with the full DVD rips uploaded. Can slowly upload it to mega for you if you wish. Just post lots of build porn later lol

I'll PM you about that DVD! Which one is your favorite?
 
I'll PM you about that DVD! Which one is your favorite?
Gosh that's a tricky question.. computer one is always a classic, the australian one is absurd, medicine one and the indian camping one come to mind but really.. they're all hilarious in their own way for different settings.
 
I’m trying to decide on a HOST os. Originally, it seemed like ProxMox VE would be a good choice as it has tons of the feature I need. However, ProxMox VE 5.3 is based on Debian Stretch 9.61, kernel 4.15.18, QEMU 2.12.1, and ZFS 0.7.12 . I have read that for threadripper, I really want a kernel that is 4.18 or later and a recent version of QEMU (3.0 or later, preferably with the PCI-E link speed negotiation patch). From what I have read, I will not be able to easily update the Kernel in ProxMox VE 5.3 (at least with my skill level) but I’m not sure if I could update QEMU. Going down this rabbit hole, it seems like ArchLinux would be my best bet for a Host OS that is properly updated. What do you think?
 
Back
Top