Hardest component to pick in a massive Home Storage Solution

Discussion in 'SSDs & Data Storage' started by kandrey89, Apr 26, 2016.

  1. kandrey89

    kandrey89 Limp Gawd

    Messages:
    182
    Joined:
    Jul 11, 2015
    Why is it that the chassis is the hardest component to pick when looking for a storage solution that contains between 24-45 disks?

    I've had all the other components picked out for several months now, but it's very hard to settle on what chassis to use, how many disks it should have, should it be backplane or direct connect, etc.

    I have a 20TB home backup storage server RAIDZ1, and need to upgrade urgently, hemorrhaging data.

    I've mulled over the following choices:
    NORCO RPC-4224 (24 disks)
    Supermicro 24/36/44/45 disk chassis in server or JBOD configuration
    Storinator (backblaze pod) 30/45 disks with Rocket 750 card

    Some of the problems that I can't solve:
    -price point is a pretty big factor, and it's hard to argue against reliability and safety but still
    -number of disks, not sure I would need 45 disks, might not fill up all of them by the time SSD technology or other storage technologies start becoming affordable at large scales, by the next upgrade I'd be using non-SATA3 drives and this chassis would not work...
    -power, paying for electricity is a significant expense per month ~$40, JBOD or server, cannot run multiple servers with multiple storage pools due to $$ and split archives
    -direct attach vs backplane, direct seems more convenient and safer, no backplane to deal with but harder to work with if a cable goes wrong. NORCO backplane might not be very reliable, Supermicro backplane should be reliable especially with the redundant backplane components but expensive...

    I'm hoping to get 8TB WD Reds, 8-12 hdds for the first round to start, should be sufficient for a few years. I also have 6 4TB disks in another server that might make sense to migrate into the new server to save power, or should I use existing server to control a JBOD, that would save ~$800 worth of various components.
     
  2. m1abram

    m1abram 2[H]4U

    Messages:
    3,175
    Joined:
    Mar 15, 2002
    I would stay away from NORCO. They are much cheaper for a very good reason. I love my Supermicro and considered the NORCO for a while before deciding on the Supermicro. Biggest factors for me was I want redunandant PSU and the Supermicro comes with it, NORCO did not and that alone brought the prices much closer. I wanted a JBOD SAS backplane and Supermicro did not have the reported issues that NORCO has. In short if you search NORCO you will find many quality control issues and once you add in the cost of a good PSU the prices are not that far apart.
     
  3. kandrey89

    kandrey89 Limp Gawd

    Messages:
    182
    Joined:
    Jul 11, 2015
    Yeah, that's what I've seen for NORCO JBODs, they're only a few hundred cheaper than Supermicro, but like a $1000 more than a non-JBOD NORCO. Doesn't make sense, $400 for a 24 bay NORCO PC case without PSU or $1500 for the same thing with a PSU and a backplane with external connectors (JBOD). Don't believe the extra JBOD and PSU add up to a $1000.

    Problem with going with PC case route is that you'll spend extra on PC components up to $800, and then pay the utility bill, (ignoring solar comments) which is like paying for a 200W light bulb 24/7 per month.
     
  4. m1abram

    m1abram 2[H]4U

    Messages:
    3,175
    Joined:
    Mar 15, 2002
    A good redundant PSU and the backplane are not exactly cheap. The PSU will run you around $500 by itself easily. Backplane though should not be more than $100 at most. So that gets you within $400.
     
  5. olavgg

    olavgg Limp Gawd

    Messages:
    232
    Joined:
    Oct 27, 2010
    Supermicro chassises are today available for less than Norco on Ebay! And they come with two PSU's.
     
  6. CombatChrisNC

    CombatChrisNC [H]ard|Gawd

    Messages:
    1,136
    Joined:
    Apr 3, 2013
    Yea, I picked up a 36 bay supermicro server for $600.

    Dual quad L5420, 32gb RAM, dual PSU, and full of disk trays.

    Added an M105 and done. Still only have 10 drives populated, but with 36 bays I have more than enough room to grow at 4-6TB per drive.
     
  7. kandrey89

    kandrey89 Limp Gawd

    Messages:
    182
    Joined:
    Jul 11, 2015
    Yeah, I'm leaning towards staying away from NORCO.

    Ebay seems to have the old SAS1 backplanes which seems like it's going to be a bottleneck, I want SAS2. Not that many choices on ebay though.

    Anyone have a guide to interpreting Supermicro's model numbering system? Their retarded site doesn't have a good comparison tool and so you have to open the ones you think you want and visually compare the differences and even then it's not clear, so you have to open the manual and look for it there, but even then sometimes it's not explicitly stated so you have to guess.
    Interested in - Super Micro Computer, Inc.
     
  8. olavgg

    olavgg Limp Gawd

    Messages:
    232
    Joined:
    Oct 27, 2010
    You can pick A or TQ. 3 controllers performs better than a SAS2 expander. And 3 pcs of for example Dell H310 is only $150.
     
  9. virtualheretic

    virtualheretic Limp Gawd

    Messages:
    237
    Joined:
    Sep 6, 2005
    Dont forget Craigslist, I bought a 16bay supermicro chasis for $75 that included fans and both PSUs
    Granted I trolled CL for 3 months before I found this deal.
     
  10. Master_shake_

    Master_shake_ [H]ardForum Junkie

    Messages:
    10,419
    Joined:
    Apr 9, 2012
    I thought this too.

    i went from a 20 port sas2 expander to a 36 sas expander and the results with 16 2tb drives were identical.
     
  11. kandrey89

    kandrey89 Limp Gawd

    Messages:
    182
    Joined:
    Jul 11, 2015
    How were you running the drives in a ZFS? Individually?

    3 PCs? No thanks, which part of the power bill didn't you get? Why would I pay 400W extra in CPU/MOBO/RAM for the 2 computers, that's like $50 per month.
     
  12. kandrey89

    kandrey89 Limp Gawd

    Messages:
    182
    Joined:
    Jul 11, 2015
    Found a naming convention explanation, though the X special features will still need to be deciphered.
    Super Micro Computer, Inc. - Product Naming Conventions

    Has anyone used supermicro chassis in their bedrooms? Can they be, assuming I replace all the fans? Is the PSU fans loud, because can't replace those.
     
  13. ST3F

    ST3F Limp Gawd

    Messages:
    181
    Joined:
    Oct 19, 2011
  14. Blue Fox

    Blue Fox [H]ardForum Junkie

    Messages:
    11,697
    Joined:
    Jun 9, 2004
    3 pieces, not PCs. The H310 is a controller.
     
  15. Master_shake_

    Master_shake_ [H]ardForum Junkie

    Messages:
    10,419
    Joined:
    Apr 9, 2012
    nope RAID 6
     
  16. westrock2000

    westrock2000 [H]ardForum Junkie

    Messages:
    9,161
    Joined:
    Jun 3, 2005
    Do you need 24 bays for expanding past 20TB? Drives will grow over time just as they always have.

    20TB is like 5 drives now a days.
     
  17. kandrey89

    kandrey89 Limp Gawd

    Messages:
    182
    Joined:
    Jul 11, 2015
    Yes, I need 24 or 36 drive bays.
    I'll initially start out with ~10 drives +-2, depending on the total # of drive bays and RAIDZ# I choose.
    WD Red 8TB, are looking good right now.
     
  18. CombatChrisNC

    CombatChrisNC [H]ard|Gawd

    Messages:
    1,136
    Joined:
    Apr 3, 2013
    I wouldn't put a supermicro in my bedroom. I wouldn't even put it next to my bedroom. You're talking 7k rpm 80mm fans, and even when in the low speed setting are doing 2k+. That doesn't take in the PSU fans either.

    And I rewired the fans in the rigs I have to pull power off the mobo since you couldn't control speed from the backplne. I think so.eone said they fixed that in newer editions.

    They also make a quiet or super model of their boxes, but I have no experience with those.
     
  19. Blue Fox

    Blue Fox [H]ardForum Junkie

    Messages:
    11,697
    Joined:
    Jun 9, 2004
    The 24 bay Supermicro chassis aren't that bad if you have one of the more modern models and high efficiency PSUs. The 36/45 bay ones will never be quiet though.
     
  20. kandrey89

    kandrey89 Limp Gawd

    Messages:
    182
    Joined:
    Jul 11, 2015
    I mentioned that I would replace all the fans with Noctua models regardless, PSU fans though are not replaceable without voiding the warranty and opening up the case, plus they typically use non standard fans in server PSUs.

    Actually the more I look into backplanes the more dissuaded I get by the price and complexity. Going to look for a direct connection backplane without expanders. Expanders increase the price of the system 2-4 fold.
     
  21. kandrey89

    kandrey89 Limp Gawd

    Messages:
    182
    Joined:
    Jul 11, 2015
    Unless you can get one from ebay for at least half the price!

    4U Supermicro 45 Bay SAS SATA JBOD Storage Expander LSI 9200-8e SC847E16-RJBOD1
    4U Supermicro 45 Bay SAS SATA JBOD Storage Expander LSI 9200-8e SC847E16-RJBOD1

    It's a JBOD, which is nice because it saves me $600-$800 for CPU, RAM, MOBO, HBA Cards, OS-SSD and it has 2 redundant power supplies (90%+ efficiency) already.
    It's a SAS2 (6Gbps) backplane, comes with HBA card and cables for the JBOD.
    It'll be a great expansion for my existing storage server without too much overhead.
     
  22. chansearrington

    chansearrington n00b

    Messages:
    2
    Joined:
    May 13, 2016
    I'm going through this same decision process right now and (pending one question in another thread) I've decided to go with:

    Norco RPC-4224
    LSI 9260-8i
    HP 12Gb SAS Expander (769635-B21)
    16x 8TB WD Reds

    Mainly because I have all the other parts and the extra electricity savings of "server hardware" isn't worth the couple extra dollars a month.
     
  23. kandrey89

    kandrey89 Limp Gawd

    Messages:
    182
    Joined:
    Jul 11, 2015
    Using a JBOD chassis:
    I don't have MOBO+CPU+RAM=~150 Watts idling, that's like >$20 per month that I don't have to pay for. Certainly makes JBODs more attractive than a server case with disks.
     
  24. JoeComp

    JoeComp [H]ard|Gawd

    Messages:
    1,036
    Joined:
    Jan 23, 2012
    The little high-speed fans on those Supermicro PSUs sound like vacuum cleaners to me. If you don't mind the noise of a vacuum cleaner running constantly, then no problem.

    I have a Norco RPC-4224 that is about 2 years old now. The main reason I got it is because it holds a lot of HDDs and allows you to easily use a standard consumer PSU (you can jury-rig a 4U Supermicro case to use a standard consumer PSU, but I was not interested in that). I have not had any problems with the backplanes, but I agree that the backplanes are a possible point of failure.

    If you want a case that can hold a lot of HDDs but does not have any backplanes, you could look at the cases from mountainMods. Their Ascension case could hold 38+ HDDs. Of course, you would need to have a SATA cable and a power cable for every HDD. And you would need a motherboard with a lot of PCIe slots for the cards to hook up to all those SATA cables.

    MountainMods.com-Computer Cases-Ascension-Ascension (Black Wrinkle Powder Coat) - Original Top
     
  25. kandrey89

    kandrey89 Limp Gawd

    Messages:
    182
    Joined:
    Jul 11, 2015
    I thought the 45 bay chassis was going to weight like 30-35 lbs, when I saw the UPS shipping weight I thought it must be wrong.
    UPS dropped it off at my door, there's a hole in the box, and it dented the chassis, I was upset but it's not that bad, it's on the underside and in the middle of nothing and I could probably push it back out.

    Anyway, I go to pickup the box and holy mother of cows, it's really 80 lbs and I huff and puff it through the door... Not sure how the UPS guy carried it 100 feet LOL.

    I go to turn it on and I think I'm hallucinating because I suddenly feel like I'm on an aircraft carrier. ROFLMAO
    I took out the trays and no wonder there's so much loud noise, the holes through the backplane comprise no more than 10% total area, holes are smaller than 1 square inch.

    I wonder if I bit off more than I can chew, I can see how it could generate a lot of heat and if it's not in a cooled room, it could really heat up... :S
     
  26. m1abram

    m1abram 2[H]4U

    Messages:
    3,175
    Joined:
    Mar 15, 2002
    Interesting that your chassis did not get shipped on a pallet. My Supermicro was shipped on pallet and yes it is heavy. I actually had to ship it to my office since the shipper required truck dock. I got it home in a pickup. As for the holes, you do not want to modify them most likely they design the airflow in the chassis fairly well. In fact make sure you use the drive blanks for empty drive slots otherwise you will not get proper airflow across the drives.
     
  27. kandrey89

    kandrey89 Limp Gawd

    Messages:
    182
    Joined:
    Jul 11, 2015
    It shipped from an ebay seller, in fact I happened to request local pickup but they accidentally shipped it.

    I can't find supermicro drive labels for 1-45, only 1-24 that comes with screws. Which reminds me, I had to buy 200 screws because the chassis is used and didn't come with screws. FYI, the reason I want drive labels is I think they pulled out the caddies at the ebay seller's warehouse or at the previous owner and did not put them back in order, so I have a bunch of caddies with the same drive numbers.

    Since I won't be populating all the drive trays at once, I might disable the power to the 2nd rear backplane, and some fans and look for alternative fans to save power and quiet in my house, it really is meant for a datacenter where noone cares that you have 7 fans running 7K RPM and generating 65dB each.
     
  28. m1abram

    m1abram 2[H]4U

    Messages:
    3,175
    Joined:
    Mar 15, 2002
    I would not disable the fans, backplane sure but fans not sure about.
     
  29. kandrey89

    kandrey89 Limp Gawd

    Messages:
    182
    Joined:
    Jul 11, 2015
    No I was thinking of disabling maybe half the fans. Each fan is 7.2W, that's $1.15 per month in electricity for an almost empty case with 1 disabled backplane. There's 7 fans in there, I'd disable 3.

    I looked at the fan enclosure and it won't accept 25mm wide into 38mm wide enclosure, there's a depression in the outer casing that would prevent the 25mm fan from being inserted. It sucks, but maybe the fan that's 45dB instead of 65dB, FAN-0074L4, might need to remove it out of it's housing but the fan dimension is the same, it's like $25 though, expensive...
     
  30. JoeComp

    JoeComp [H]ard|Gawd

    Messages:
    1,036
    Joined:
    Jan 23, 2012
    Have you actually measured the power consumption of the fans with a meter?
     
  31. kandrey89

    kandrey89 Limp Gawd

    Messages:
    182
    Joined:
    Jul 11, 2015
    You know what, I will since you ask, today.
    Why do you ask, you think the rated current on the fan is just for show? In other fans at work, they are ~80-90% of rated current.

    Although the current goes up with higher applied resistance, when blowing inside a tight enclosure, when static pressure is increased, so a free standing air test will give the minimum.
     
  32. JoeComp

    JoeComp [H]ard|Gawd

    Messages:
    1,036
    Joined:
    Jan 23, 2012
    I just thought 7W sounds like it is at the high end, like you say with high static pressure, and also full speed. I wonder if the power usage is 7W in your actual server. Also, I don't know if you have a supermicro motherboard, or some other motherboard that allows you to adjust the fan speed -- if so, it would be interesting to see how the power draw changes when you slow the fans down. I bet the power draw goes down significantly at lower RPM.
     
  33. tschopp

    tschopp n00b

    Messages:
    15
    Joined:
    Oct 13, 2011
    I have a supermicro chassis next to a bedroom. If you plug the fans into a mb to reduce the speed they won't be noisy. I would not replace them, you need fans that can make pressure, just reduce the rpm until the sound is OK. Not sure if you can fix the power supplies. I replaced the power supplies with a normal power supply. You should also fill any holes in the fan wall with some foam, this will help with airflow so you can turn the fans down more.
     
  34. SomeGuy133

    SomeGuy133 2[H]4U

    Messages:
    3,447
    Joined:
    Apr 12, 2015
    Norco 4224 is a decent case for none pro needs. Just test all the drive bays and make sure the case isn't bent or has any issues. Mine was bent in several places and due to time i did not have the time to RMA it and get a new one so i took a hammer and beat the shit out of it so it would work. This goes for any case but fair warning.

    fans range from 1-2w to 40w. I have high power fans that use over 36w.

    Static pressure is only needed on heat sicks or spaces with little room. Static pressure provides less air flow but force to push air through openings like heatsinks are cluttered cases.

    Norco 4224 case does not need high static pressure fans. Standard ones work fine in that case. small ITX or heatsinks high static pressure is good.

    Delta PFB0912DHE-F00 Case Fan - Newegg.com
    Delta AFB1212GHE-CF00 Case Fan - Newegg.com

    there are even stronger fans lol

    thats 63 cents where i live...you must have terrible energy prices.
     
    Last edited: May 20, 2016
    trumee likes this.
  35. JoeComp

    JoeComp [H]ard|Gawd

    Messages:
    1,036
    Joined:
    Jan 23, 2012
    That is nonsense.

    Static pressure refers to the pressure difference "across" a fan when there is no net air flow through the fan. For example, if you seal a fan to an opening in an otherwise airtight box, then run the fan at full voltage. The pressure difference between inside the box and outside is called the "static pressure".

    Some fans are capable of producing a higher static pressure than other fans, and usually these fans have lower airflow (when airflow is not restricted) than fans that are not capable of producing high static pressure. Additionally, if a fan is capable of high static pressure, it is also usually capable of high pressure at low (but not static) airflow through the fan.

    Perhaps what you meant to say is that fans capable of producing a high static pressure are useful on heatsinks. While that may be true, it is not the only use for fans capable of high static pressure. Rackmount cases with dense HDD hot-swap trays often restrict the incoming airflow significantly, so it is useful to have the fans right behind the backplane be capable of high static pressure.

    But really all of this is a digression from what kandrey89 and I were talking about. kandrey89 alluded to the static pressure of the fans affecting the power consumption. I believe what he meant is if the airflow to the fans is restricted, the fan will need to overcome a large pressure differential from intake to exhaust and this will create a higher load on the fan, thus increasing power consumption. I agreed, and also mentioned that speed (RPM) will also affect power consumption.
     
  36. SomeGuy133

    SomeGuy133 2[H]4U

    Messages:
    3,447
    Joined:
    Apr 12, 2015
    You really didnt read what i said. How was that complicated.

    not ridiculous that's what static pressure is and why you use them on heat sinks. Its their ability to push air in small spaces due to the pressure they can make. Its about the blade design. A high static pressure fan has lower CFM but better pressure. A low static pressure fan has better CFM. They have difference uses.

    I said high static pressure is good for small cluttered cases like ITX, heatsinks, and where there is little free open room like some hard drive racks. The norco 4224 has plenty of space and static pressure fans are a waste. Here is the case i mentioned and you can see static pressure fan would be mostly pointless.

    Dropbox - 2016-05-21 08.50.46.jpg

    http://direct.z-95.com:8888/pics/ZS12/serverinside2.jpg
     
  37. kandrey89

    kandrey89 Limp Gawd

    Messages:
    182
    Joined:
    Jul 11, 2015
    The point is, the fan draws ~580mA without any obstruction, in free standing air, standalone. Obstructing the fan on both sides may increase the current by no more than ~150mA.
    At 8V, the fan draws about half as much, but the problem is if you've been reading what case I got is that there is no mobo, there's only the backplane and power distribution card.
    I'm going to check if my Noctua fan PWM cables could do something about it. I have a bunch of attenuation cables from previous purchases that could bring down the voltage, will see. Not sure if they're rating for 3 times the current compared to Noctua fans though :D
     
  38. JoeComp

    JoeComp [H]ard|Gawd

    Messages:
    1,036
    Joined:
    Jan 23, 2012
    Actually, fans capable of high static pressure are an excellent choice for the fan wall of the Norco 4224, since the airflow through the HDDs sleds is restricted. In order to get significant airflow through the restricted space, it is necessary to create a low-pressure region between the fanwall and the HDD sleds. This means that there will be a significant pressure differential across the fan wall, and fans capable of producing such a pressure differential at relatively low RPMs are desirable if you want good airflow and low noise.

    I'm not sure why you are showing me pictures of a Norco 4224. Did you not read my post where I said I have one?
     
  39. JoeComp

    JoeComp [H]ard|Gawd

    Messages:
    1,036
    Joined:
    Jan 23, 2012
    Why not get a fan controller? Then you can adjust the voltage to multiple fans easily. And it should specify the currents that it can handle.

    Although I am not sure whether that will decrease the power consumption a lot, since there will be some current going through a resistor (it will reduce the overall current through the voltage divider, but there will still be some unwanted power dissipation in the resistor). Ideally, you should reduce the voltage at the supply. Like using 5V instead of 12V. But a lot of fans will not spin at all, or spin very slow at 5V.
     
    Last edited: May 21, 2016
  40. SomeGuy133

    SomeGuy133 2[H]4U

    Messages:
    3,447
    Joined:
    Apr 12, 2015
    Show me any documentation proving static pressure fans are needed for areas of 2-3 inches of open air? with a vast area of open air on the other side. Static pressure fans should not be needed for gaps that large. (true static pressure fans) Real static pressure fans can have half the airflow. If your saying a hybrid fan would be better suited for that sure.

    diminishing returns here. Disabling 7 W fans is ridiculous unless your running on Solar or paying 50 cents a kilowatt. If its that big of a deal get better fans. Particularly ones with senses that spin slow but spin faster when air temp reaches a certain amount or put a controller that controls themp based off OS readings.

    If you reduce voltage you will ruin efficiency. voltage is more efficient than currant. Also fans have certain speeds where they are most economical to run. Performance per watt if you want to call it that. Your really wasting your time with this. I have never heard anyone do this before besides if their running on back up power or have a certain power envelope in a data center to stay under.