Building a cluster computer?

E

enkyrpted

n00b
Joined
Jun 27, 2013
Messages
4
I'm sorry if this is the wrong place to ask this, but what I want to do is make a 32 core cluster computer.
I want to use one master and the other as nodes. So I want the storage to be centralized, and the operating system running on only one machine.
I want to use Windows 7/8 on this machine, and use it for rendering 3d movies and photoshop work. I also want to add a graphics card and 16 gb ram to each of the 4 motherboards, and they all should utilize load balancing, and ideally, power up only when their resources are needed.
I read about beowulf, but a beowulf cluster uses identical hardware, which I dont want to. Also, I want it to utilize as much CUDA or OpenCL as possible.
I want the computer to identify all the hardware as one single piece if possible, but I want to power up each node only when the node before it reaches 75% usage.
I haven't done any work using clusters before, so I barely have any knowledge of the field.
So can someone please tell me whether or not this is possible? And what kind of a cluster would need to be made for this? Thanks in advance.
 
Moved to the Networking & Security since you might get better help here.
 
You're over thinking the problem as far as I can tell, a real cluster would be beowulf (architectures need to match but not precise hardware afaik); but you're only really needing to offset processing?

Most 3d rendering software suites have a remote rendering feature which you can install on another machine, either way you can get most of the hardware specs you mentioned into a single workstation....
 
I want 4 AMD processors, 4 Amd firepro 6 GB cards, and 8 pcie 64 gb ssd's. I dont think that would fit into a single workstation. architecture needs to match as in?
 
enkyrpted said:
I want 4 AMD processors, 4 Amd firepro 6 GB cards, and 8 pcie 64 gb ssd's. I dont think that would fit into a single workstation. architecture needs to match as in?
Click to expand...

I know you can put all of that in a server, possible exception of PCIe SSDs. RAID SAS would give similar performance and would easily fit in a case containing the rest. You may need to bump to Server 2k8R2 Enterprise/Datacenter or 2K12 to utilize all of the processors at once.
 
OP, You say that you want specific hardware and then don't understand how to configure/arrange it? No offense, but in the first post, I see a bunch of buzzwords without much organization. Let start with the basics, with more detail please:

1) What are you trying to accomplish?
2) What software will you be using for processing of Data?
3) What kinds of data will you be working with? Type/sizes/demands/etc
4) Assuming large file sizes, what storage capacity will you be needing?
5) What is the budget for the project?

As RocketTech said, you can get a Server with enough sockets to run your entire list of hardware in the same box.
Is this for Personal lab environment or production use?
 
1) I want to specifically build a cluster, beowulf or the like, and I know I will have to use infiniband for it.
2) I will be running photoshop, sony vegas, adobe after effects, adobe premier, Audacity, and more.
3) I need to render around 20 GB sometimes in after effects, Vegas and premier is around 8-10 GB each. Not that my current computer can't handle it, but it takes a long time, and I need it to be super fast.
4) I will be adding, at the start, 2x4 TB Sata 6 gbps drives and 8 pcie 64 gb ssd's for caching, but now that I think about it, I guess I need to bring that number down to 4.
5) My budget is around $10-20k.
Its for a personal lab environment.
 
you're not going to be able to do this at an OS level with windows 7/8....

you need to be checking out the work offloading capabilities of your applications....

there will be no EASY way of doing this, besides fitting everything on a single motherboard...
 
How about linux? And I must have to install some software before the app is able to offload, right?
How difficult can it be?
 
enkyrpted said:
How about linux? And I must have to install some software before the app is able to offload, right?
How difficult can it be?
Click to expand...

What about Linux? Virtually most of the software you're talking about only runs on Windows or Mac. Also, can you give us the FULL list of software that you're planning on using? "And more' is too open-ended of an answer.

As for how difficult this project can be, it can be relatively difficult if you can't grasp how some of the software works or think out what exactly you're trying to do. In this case, some of the software you've listed don't have network options (Audacity, Photoshop, Premier (maybe))

What are the specs of your current PC?
 
I have a couple of 4Ps with AMD 16 core and 12 core processors that I mounted in tall gaming towers. You will have a problem finding server boards with enough PCIe slots to run 4 video cards. You could possibly run two Titans or similar. However, I'm not sure the bus is going to work well for that or not. The other problem is finding a PSU that is strong enough to handle it all.
 
You can get 4 PCIe slot server MoBos pretty easy. It seems OP is just tossing hardware names around as stated before and no real plan to integrate for the stated purpose.
$20K also seems kind of tight for a 4-Proc Mobo, 4 FirePro W-series, SAS SSDs, RAID Card, and Procs.
If Software needs to be included in the budget, forget it- you blew the budget.
The biggest question is will the software see any actual performance increase when run on 64 cores vs 4-8? If you were running crysis while rendering a multi-terrapixel image and streaming spotify, then yes, no problem, poor it on. If you expect 64 cores to drop your render time compared to a 4-core machine by a factor of 16- forget it.
 
RocketTech said:
You can get 4 PCIe slot server MoBos pretty easy.
Click to expand...

Sure, pcie, but not suitable for 4 double width x 16 that you'd need for a workstation graphic card. 4P boards are huge but AMDs typically have 8 memory slots per CPU which doesn't leave a lot of room when you have 32 and 4 1944 pin sockets for the CPUs.

BTW, the most x16 slots of any type or spacing forva 4p AMD G34 I've heard about is only 3.

I do agree that a 4 p might not be the way to go. An over clocked 2p Intel 6 or 8 core with the same GPUs may be close if not faster. My 2p 16 core is slower ripping than my 4 core Intel that is over clocked. Not quite rendering though.
 
Last edited:
You must log in or register to reply here.
Back
Top