![[H]ard|Forum](/styles/hardforum/xenforo/logo_dark.png){kind=logo}
pbandstefanwich
n00b
- Joined
- Jun 5, 2012
- Messages
- 4
This is my first big storage system build, and I’d like some advice and input before I start setting up everything. I’ve been doing research and buying some stuff on and off over the past couple months. Changed my plans a couple times, sold a bunch of stuff, etc. So now I’d like to have a firm plan before I buy everything else! I’m setting up two systems: a main storage server and a HTPC. I have grouped them into one thread because I intend for them to be fairly interconnected, as I’ll discuss later.
Some background on me: I’ve used Windows for many years, have been using Mac for the past five years, and have some decent Linux experience. I am by no means a Linux expert, but not a newcomer either. I don’t have much experience with Unix, but am of course willing to put in the time to learn whatever is necessary.
OK, so now on to the build and questions I’ve got.
SERVER:
This will primarily be a storage server. I would like to store all my music (over 2TB flac), pictures, data, time machine backups, and videos on there. I am going to start doing a lot of Blu-Ray rips, so those will take up a good deal of space to. I sometimes do work in MatLab, Python, and R, so I might send some jobs to the server to complete. But the vast majority of the time, this will function as a data/media storage server.
1.) Case: I have decided I need a full tower consumer case. I thought about server cases like the Norco, but I need the case to be as quiet as possible (I don’t have a server closet). I don’t need it be super pretty, but I need it to look acceptable tucked away in the corner of say, a living room. Thus, I think the Fractal Design Define XL seems appropriate. 10 drive bays, then I will add a 4-in-3 module to expand the capacity to 14 3.5” drive bays and one DVD drive. Seem like a reasonable pick?
2.) OS: I have not fully decided on this one yet. I’ve been flip flopping between FreeBSD and OpenIndiana. I know I want to have ZFS, so I suppose Nexenta is another option. I haven’t seen any general consensus on which of the two (OI or FBSD) is better, they’re more different flavors? Would there be any compelling reason to pick one over the other? I tried OI 151a a little while back, and some parts seemed rather buggy to me. I have never tried to use FreeBSD before, but I thought about giving a try with the KDE version (I will use a lot of CLI, but the GUI is nice option to have too). Also, the support group behind FreeBSD seems much larger. When I tried OI, I couldn’t even figure out how to do a successful install to a USB key! Then I couldn’t figure out how to set it up to auto login upon system boot, etc.
3.)Processor: I have decided on either the Xeon E3 or E5 series. The main deciding factor will be how much ram is necessary. In the end, I am looking to have 16TB of usable space. Would 16GB of ram be enough? I am inclined to say that 16 is enough, and I should go with the Xeon E3-1230v2. If I will need more than 16gb, I will probably opt for an E5-1620. It is significantly more expensive, and draws more power, but 8GB ECC UDIMM are insanely expensive (if I can even find one).
4.)Motherboard: I haven’t decided on a specific model, but I am going to go with a SuperMicro, most likely. I have kind of put off deciding which precise model until I decide on a processor.
5.)Ram: 16GB Kingston UDIMM with the E3, or somewhere between 32 to 64 GB with the E5
6.)HDD: I am going to start off with 6x2TB drives in a RAIDZ2 config. Later I will add another identical pool to bring the storage to 16TB total. I have been buying HDDs whenever I see them on sale, so they are different manufacturers (I have some Seagate and WD right now). I am assuming this won’t matter that much? Ideally they’d all be 7200RPM drives, but I don’t think I need that because this will be primarily a media and data storage server and speed won’t be much of an issue.
7.)PSU: I currently have both an Antec Platinum 550W and a Seasonic Bronze 650W. I know Seasonic typically has a better reputation, but I am inclined to use the Platinum one instead for a 24/7 server. Any input?
8.)CPU cooler: I was thinking about picking up a Noctua, but haven’t made up my mind yet. Any suggestions? I won’t be doing anything super processor intensive, but I want to keep the processor as cool as possible, and be quiet at the same time.
9.)SATA cards: I think most, if not all the motherboards come with around 6+ sata ports. I have two RocketRaid 4-port PCIex4 cards that I was planning on using in JBOD mode, so I should have at least 14 SATA ports
HTPC:
Pretty much self-explanatory, however I was thinking about putting a small drive pool in this one to back up important files off the main server.
1.)Case: I have a Moneual Moncaso 932B I am going to use. It is fairly large, but it’s an awesome case. Has room for 4 3.5” drives. My OS will be on a SSD, so I’m sure I can find somewhere to Velcro it down.
2.)OS: Currently just windows 7. However, I have considered making this an ESXi server with windows 7 and either OI or FreeBSD with a small pool to serve as a backup for the main server
3.)Processor: I would love to have ECC on all my systems, but it’s not necessary on the HTPC. I was going to go with an ivy bridge i5, but if I’m going to be backing up my critical data on here, I re-thought the ECC issue. Anyone think ECC will be necessary in this case? If so, I will likely have to get either a Xeon or an AMD.
4.)GPU: This is another reason I wanted the ivy bridge. The iGPU should have been enough for this. The most intensive thing I would do is 3D blu-ray playback. However, if I end up wanting ECC, I’ll have to go with a Xeon or AMD, in which case I might have to get a discrete GPU.
5.)HDD: I have 4x1.5TB drives I was thinking about putting in here in a RAIDZ1 config.
Misc thoughts/questions:
1.)What do you guys think about having my HTPC serve as a backup for the server? I know I need some type of backup just in case the server goes up in flames or something like that. I know it should ideally be off-site, but I don’t have that capability at the moment. I considered a tape drive backup and storing the tapes in a fire/flood proof safe, but there’s got to be a better and more robust solution.
2.)If using HTPC as a backup drive, should I go the hypervisor route and install both windows and OI/FBSD? Im thinking that would be the best route so I can have ZFS on there too, I just don’t know how well ESXi would play in a HTPC role. The main issue I foresee is the PCI passthrough. I would need to pass through a bunch of tuner cards, and the GPU. If I was using the integrated HD4000 graphics, is that even possible to pass through to Windows through ESXi?
3.)I have also played around with the idea of virtualizing my main server via ESXi. Then I could play around with some routing and firewalls, media server OS, etc. However, I had bad experiences with XCP when I tried it. It was a royal PIA to even get OpenIndiana installed and running in the first place. Then the PCI passthrough was a nightmare and I gave up on it. Is ESXi easier? The other issue is that I remember reading ESXi will increase the power draw, even at idle with only one OS running. This is something I don’t want… I want the server to be as low power consumption as possible if it’s going to be on 24/7.
4.)HDD spin up/spin down (feel like I’m talking about electrons here). Are there options within FreeBSD or OI about whether or not to spin the HDDs down when not in use? I’m not sure the benefits/risks associated with this. It would probably save a little bit of power, but I’m also worried about the wear and tear on the drives from starting and stopping the platters so many times. Another option would be to include a sizable SSD cache, and have the HDDs set to spin down when not in use. Hopefully the SSD cache would really reduce the number of times the HDDs would have to spin up?
Phew, sorry for the huge wall of text. I’ve had so much on my mind and so many questions have surfaced over the past couple months I needed to get it all out! Thanks for reading and thanks in advance for the help guys.
Some background on me: I’ve used Windows for many years, have been using Mac for the past five years, and have some decent Linux experience. I am by no means a Linux expert, but not a newcomer either. I don’t have much experience with Unix, but am of course willing to put in the time to learn whatever is necessary.
OK, so now on to the build and questions I’ve got.
SERVER:
This will primarily be a storage server. I would like to store all my music (over 2TB flac), pictures, data, time machine backups, and videos on there. I am going to start doing a lot of Blu-Ray rips, so those will take up a good deal of space to. I sometimes do work in MatLab, Python, and R, so I might send some jobs to the server to complete. But the vast majority of the time, this will function as a data/media storage server.
1.) Case: I have decided I need a full tower consumer case. I thought about server cases like the Norco, but I need the case to be as quiet as possible (I don’t have a server closet). I don’t need it be super pretty, but I need it to look acceptable tucked away in the corner of say, a living room. Thus, I think the Fractal Design Define XL seems appropriate. 10 drive bays, then I will add a 4-in-3 module to expand the capacity to 14 3.5” drive bays and one DVD drive. Seem like a reasonable pick?
2.) OS: I have not fully decided on this one yet. I’ve been flip flopping between FreeBSD and OpenIndiana. I know I want to have ZFS, so I suppose Nexenta is another option. I haven’t seen any general consensus on which of the two (OI or FBSD) is better, they’re more different flavors? Would there be any compelling reason to pick one over the other? I tried OI 151a a little while back, and some parts seemed rather buggy to me. I have never tried to use FreeBSD before, but I thought about giving a try with the KDE version (I will use a lot of CLI, but the GUI is nice option to have too). Also, the support group behind FreeBSD seems much larger. When I tried OI, I couldn’t even figure out how to do a successful install to a USB key! Then I couldn’t figure out how to set it up to auto login upon system boot, etc.
3.)Processor: I have decided on either the Xeon E3 or E5 series. The main deciding factor will be how much ram is necessary. In the end, I am looking to have 16TB of usable space. Would 16GB of ram be enough? I am inclined to say that 16 is enough, and I should go with the Xeon E3-1230v2. If I will need more than 16gb, I will probably opt for an E5-1620. It is significantly more expensive, and draws more power, but 8GB ECC UDIMM are insanely expensive (if I can even find one).
4.)Motherboard: I haven’t decided on a specific model, but I am going to go with a SuperMicro, most likely. I have kind of put off deciding which precise model until I decide on a processor.
5.)Ram: 16GB Kingston UDIMM with the E3, or somewhere between 32 to 64 GB with the E5
6.)HDD: I am going to start off with 6x2TB drives in a RAIDZ2 config. Later I will add another identical pool to bring the storage to 16TB total. I have been buying HDDs whenever I see them on sale, so they are different manufacturers (I have some Seagate and WD right now). I am assuming this won’t matter that much? Ideally they’d all be 7200RPM drives, but I don’t think I need that because this will be primarily a media and data storage server and speed won’t be much of an issue.
7.)PSU: I currently have both an Antec Platinum 550W and a Seasonic Bronze 650W. I know Seasonic typically has a better reputation, but I am inclined to use the Platinum one instead for a 24/7 server. Any input?
8.)CPU cooler: I was thinking about picking up a Noctua, but haven’t made up my mind yet. Any suggestions? I won’t be doing anything super processor intensive, but I want to keep the processor as cool as possible, and be quiet at the same time.
9.)SATA cards: I think most, if not all the motherboards come with around 6+ sata ports. I have two RocketRaid 4-port PCIex4 cards that I was planning on using in JBOD mode, so I should have at least 14 SATA ports
HTPC:
Pretty much self-explanatory, however I was thinking about putting a small drive pool in this one to back up important files off the main server.
1.)Case: I have a Moneual Moncaso 932B I am going to use. It is fairly large, but it’s an awesome case. Has room for 4 3.5” drives. My OS will be on a SSD, so I’m sure I can find somewhere to Velcro it down.
2.)OS: Currently just windows 7. However, I have considered making this an ESXi server with windows 7 and either OI or FreeBSD with a small pool to serve as a backup for the main server
3.)Processor: I would love to have ECC on all my systems, but it’s not necessary on the HTPC. I was going to go with an ivy bridge i5, but if I’m going to be backing up my critical data on here, I re-thought the ECC issue. Anyone think ECC will be necessary in this case? If so, I will likely have to get either a Xeon or an AMD.
4.)GPU: This is another reason I wanted the ivy bridge. The iGPU should have been enough for this. The most intensive thing I would do is 3D blu-ray playback. However, if I end up wanting ECC, I’ll have to go with a Xeon or AMD, in which case I might have to get a discrete GPU.
5.)HDD: I have 4x1.5TB drives I was thinking about putting in here in a RAIDZ1 config.
Misc thoughts/questions:
1.)What do you guys think about having my HTPC serve as a backup for the server? I know I need some type of backup just in case the server goes up in flames or something like that. I know it should ideally be off-site, but I don’t have that capability at the moment. I considered a tape drive backup and storing the tapes in a fire/flood proof safe, but there’s got to be a better and more robust solution.
2.)If using HTPC as a backup drive, should I go the hypervisor route and install both windows and OI/FBSD? Im thinking that would be the best route so I can have ZFS on there too, I just don’t know how well ESXi would play in a HTPC role. The main issue I foresee is the PCI passthrough. I would need to pass through a bunch of tuner cards, and the GPU. If I was using the integrated HD4000 graphics, is that even possible to pass through to Windows through ESXi?
3.)I have also played around with the idea of virtualizing my main server via ESXi. Then I could play around with some routing and firewalls, media server OS, etc. However, I had bad experiences with XCP when I tried it. It was a royal PIA to even get OpenIndiana installed and running in the first place. Then the PCI passthrough was a nightmare and I gave up on it. Is ESXi easier? The other issue is that I remember reading ESXi will increase the power draw, even at idle with only one OS running. This is something I don’t want… I want the server to be as low power consumption as possible if it’s going to be on 24/7.
4.)HDD spin up/spin down (feel like I’m talking about electrons here). Are there options within FreeBSD or OI about whether or not to spin the HDDs down when not in use? I’m not sure the benefits/risks associated with this. It would probably save a little bit of power, but I’m also worried about the wear and tear on the drives from starting and stopping the platters so many times. Another option would be to include a sizable SSD cache, and have the HDDs set to spin down when not in use. Hopefully the SSD cache would really reduce the number of times the HDDs would have to spin up?
Phew, sorry for the huge wall of text. I’ve had so much on my mind and so many questions have surfaced over the past couple months I needed to get it all out! Thanks for reading and thanks in advance for the help guys.