Building a computer that scales with its usage

[Kalsan]

Hi there! This is my first post and I'd like to share a dream of me with you, looking for discussion and hoping to learn lots of things. Let me say beforehand that I'm unexperienced with hardware and that everything I write here is information I collected by surfing and asking people. Many of the things I write are probably illusions and utopias and I hope to get confronted with reality(TM) in this thread.

My idea is to build a computer that scales with its usage. This means that it consists of a fast, high-power computation part (containing main CPU, GPU) and a low-power headless storage part (containing a small CPU and a few SSDs with RAID). The fast part only gets turned on when needed, so the construct can awake to a monster when required, but otherwise has low power consumption when mostly idle (still able to perform basic NAS-style tasks, e.g. non-converting DLNA server, playing back a movie, synching some data at low rate). This idea being very general, I try to be as open as possible about what a possible implementation could look like. I'd like to consider high-cost and non-conventional solutions as well, dreaming in all kinds of directions and either end up with the super-cool PC of my dreams or having at least learned lots of things.

I've head a few design ideas which I would like to present to you (feel free to destroy them but please tell me why they're irrealistic):

Idea 1: Big and Small, both self-made and Small contains the data: A big self-made case that contains two computers. The small one really is a NAS. The power consumption could be pushed down to 10W if I use Xeon D (so I heared), while still being RAID-capable. The big computer (in the same case) is the beast and can be booted (e.g. wake-on-LAN) by the NAS. Both machines are connected by a smokin' fast kind of network (maybe SFP?) and the big computer only has a system disk (user files reside on the network mounted NAS). An Arduino controlled by the NAS could even physically unplug the power supply of the big machine when it's not running in order to save power. Problems: I heared that it's hard / super expensive to get hardware that provides decent RAID support while consuming 10W only. Also it's unclear which protocol to use to mount the home partition in order to achieve minimal latency and maximal throughput.

Idea 2: Big and Small, Small is commercial and contains the data: A commercial NAS (meaning: one that I don't build myself) and a fast computer. Again, the NAS gets mounted by the computer upon boot. Problems: I've seen a few commercial NAS and none has ever made me happy. The software running on them is out-dated or even locked, and even if I manage to install some fresh Linux, the architecture is often so exotic that not all of my software runs on it. And most importantly: I've never seen a NAS that's fast enough to replace a SATA-connected SSD.

Idea 3: Big and Small, both self-made, shared external storage: Some kind of shared storage that is not a NAS and that gets shared by a big machine (the beast) and a low-end machine such as Raspberry Pi. The problem here: I have never heared of any kind of fast shared storage (with consistent simultaneous access) that is not a NAS.

Idea 4: Big and Small, both self-made, each contains part of the data: This is the capitulation plan and would probably be the classic solution: "hot" data (data I currently work on) sits on the big computer. The NAS takes the role of an archive. Problems: Need to decide what data goes where, cannot utilize storage on one computer for the other (too slow), lots of waiting for data to sync between computers, need to run the big machine to access data that hasn't gone to the NAS yet.

I'm looking forward to this discussion and I hope some of you like to dream with me. I'm also happy to get links to further reading.

 

[silent-circuit]

I think you're over complicating this. All you need is a router that supports NAS and can act as a DLNA server, that's your data backup and movie storage. That's always on, always available. If you need more power you turn on the desktop, which also has access to that data if needed but would mostly push things to the NAS, not read from it much, and will have its own fast storage.

If that won't work for you I guess I don't understand your work flow.

 

[rtangwai]

Just put 2 computers inside a case like the Thermaltake Core W200 - it even has separate power buttons for each computer.

For the NAS run FreeNAS and use a cheap controller like a M1015 flashed to LSI 9211-8i. If you need more than 8 drives in the NAS buy a SAS expander and a P200 case extension to hold more drives.

 

[Kalsan]

I think you're over complicating this. All you need is a router that supports NAS and can act as a DLNA server, that's your data backup and movie storage. That's always on, always available. If you need more power you turn on the desktop, which also has access to that data if needed but would mostly push things to the NAS, not read from it much, and will have its own fast storage.

If that won't work for you I guess I don't understand your work flow.

Thanks for your reply! There's one detail that I might have not explained clearly enough: If possible I want the big machine to be disk-less except for the system disk, meaning that all the user files (/home under Linux, "C:\Users" under Windows) reside on a central storage that is always on (i.e. the NAS). I need to be able to remote access most of my data, this is why I tend to prefer this design as opposed to a design where part of the data resides in the big computer. My hope is to get the NAS so fast that it can replace (or at least approximate) a SATA connected disk.

The fastest router I know is the turris omnia and it has NAS capability. Since I couldn't find info on how fast the thing is, I asked a friend that has one to benchmark it. If he does so, I'll post the results to let you know.

 

[Kalsan]

Just put 2 computers inside a case like the Thermaltake Core W200 - it even has separate power buttons for each computer.

For the NAS run FreeNAS and use a cheap controller like a M1015 flashed to LSI 9211-8i. If you need more than 8 drives in the NAS buy a SAS expander and a P200 case extension to hold more drives.

Thank you for your tips! I don't think I'll ever find a case that fits my needs so I'll probably have to build my own, but that's fine.

However I find your suggestion with FreeNAS and RAID controller very interesting. Did I understand this correctly that this means building a cheap computer and giving it Hardware RAID support along with FreeNAS as an operating system?

 

[rtangwai]

Thank you for your tips! I don't think I'll ever find a case that fits my needs so I'll probably have to build my own, but that's fine.

However I find your suggestion with FreeNAS and RAID controller very interesting. Did I understand this correctly that this means building a cheap computer and giving it Hardware RAID support along with FreeNAS as an operating system?

Not hardware RAID - FreeNAS uses ZFS so you would set up RAIDZ arrays. The M1015 would become a JBOD controller, I just suggest it because they are cheap, plentiful, and work very well in FreeNAS (I use one myself hooked to a chearp HP SAS expander for 24 ports).

FreeNAS itselft would be on a computer, although given the RAM requirements it may not be as cheap as you might like. FreeNAS now supports Bhyve so you can take it a step further and have FreeNAS do background tasks that your primary workstation doesn't need to do full time (download torrents, TV/video server, pfSense router/firewall, etc.).

You could also do it the opposite way - build an ESXi box and host FreeNAS as one of your VMs. Thats what I do (although I use separate computers instead of one giant case), so FreeNAS coexists with other things like my Ubuntu server and Emby media streamer. That way you can have multiple services on multiple VMs inside a single box.

You might be surprised how well the W200/P200 combination would fit your description. The W200 is a refinement of the original Everex "supercube" case design (which separated HDDs and PSUs from the electronics). Each side of the case is big enough and fitted out to hold an entire computer in its own right. The P200 extension allows you to install lots of HDDs for the NAS or other items such as watercooling reservoirs.

 

[Kalsan]

Thanks again! This sounds very interesting indeed. I'm already familiar with KVM / qemu, but I'll have to take a look at Bhyve too.

VMs certainly are an option if the host keeps running 24/7, but assuming I'd put the VM on the large computer, the setup would not allow to save power due to the high idle consumption of the host.

Clearly the W200/P200 is beautiful. The problem is that it doesn't fit into the space where the machine would be (actually mounted on the wall), which is why I think I'll have to get some wood and a saw. No predesigned case will probably ever fit that space due to its shape.

Btw a friend just showed my this fanless computer which seems very interesting for a high-performance NAS: http://www.fit-pc.com/web/products/ipc3/

 

[Kalsan]

There's yet another option, combining a kind of beast (not extremely fast but quite competitive) and a low-power idle consumption (10W) in a fanless case: https://airtop-pc.com/
This might be able to play the role of NAS and main computer in one. It has a nice idle consumption and still has support for a Xeon processor.

[EDIT]: judgment of a friend of mine: "Outdated notebook hardware, and idle consumption is most probably calculated without the loss at the transformator"