Rackable SE-3016 JBOD - Poor mans 100TB+ server

packetboy you still havn't given the full specs or the server you are using?

We can see it's got 4 gig ram... What cpu's motherboard etc?

Or if it's an all in one. How many virt cpu's did you give it etc?

For 32drives and two controllers it's going to need some horsepower and RAM to perform it's best.

.
 
It's a Gigabyte Z86 with a Intel Core i3 processor (quad core)...forget the Mhz. This was NOT specd as a storage server..it's actually my HTPC that I stole to use as a temp test machine.

We've just moved these arrays over to a second test server which is a Supermicro dual Xeon 3.0 Ghz (12 cores) w/ 48GB of ram and LSI 9205-8e HBA...we're actually running FreeBSD/ZFS...the initial numbers (only tested one enclosure) look pretty darn good (800MB write, 1100MB/s read)...will post full numbers tomorrow.

These are NOT setup in any kind of all-in-one approach.
 
hmm, i would try OI then. freebsd is slower than OI still and i would imagine the lsi drivers aren't quite as good yet.
 
madrebel said:
hmm, i would try OI then. freebsd is slower than OI still and i would imagine the lsi drivers aren't quite as good yet.
Click to expand...

Actually, the test results with the Core i3 was with OI151a5.

The server I'm building is to store high-volume traffic captures (e.g. up to 100-200MB/s 24x7). Our capture server is currently FreeBSD as it is fairly well optimized to handle high volume traffic captures (e.g. they support network adapter 'polling' mode which essentially eliminates interrupt processing).

The challenge is that we need to lay these captures down on a huge storage server. Initially were were looking at having a OI151 ZFS server that was Infiniband connected. Though FreeBSD 9 claims in now supports IB, don't believe it...it is totally unstable and not performant. Also as NFS 4.1 is not available in FreeBSD yet, there's no possibility of doing NFS/RDMA. Also, as soon as we try to enable ConnectedMode FreeBSD actually hangs/crashes...definitely NOT ready for prime time.

Then on a lark, we though that if we just did ZFS ON the FreeBSD server then we wouldn't have to worry getting some kind of high speed transport between the capture server and storage server. I'm skeptical, but we're going to give it a try as it would certainly simplify things.

BTW (as expected) it was trival to export the ZFS pool we created on OI151a5 and import it into the FreeBSD.
 
cantalup said:
Correct.

I did on mine. Has been running 24/7 for months.
If you need to lower down the PSU noise. Replace 40mm PSU fun with 3000 rpm fan. The fan that comes from PSU is running 10k rpm...
Click to expand...

I want to thnk you Cantalup for your ideas in this message. I live in Carrollton Tx not too far from one of the ebay sellers that sells these SE3016 devices. I needed something like that for my business and it turned out that the SE3016 is a great DAS device that is really inexpensive - and with the benefit for me of local pickup.

It's so good in fact, that I wanted one for home. The only problem is that it's just too loud to use in my home office. So I took your advice and got some fans. Went to Fry's and got two 80mm fans that are direct replacements for the fans in the back of the case (80mm antec tricool). That got rid of 1/2 the noise. Then I noticed how much noise the power supply was making. So I purchased a 40mm fan. Didn't really get a good match as the one I got was a silenx 40x40x10. It not only didn't blow enough air, but didn't fit in the power supply well and left a big gap. So I looked at the old fan and found that it was 40x40x28, and actually found that I had one in an old broken PC that was .3amp vs the .6amp that comes in the stock power supply with the se3016. So cut off the connector from mine, soldered on the original ps connector, fired it up, and now it's almost silent.

I'm VERY happy with this solution as while it's not 6GB, it doesn't matter much anyway if you plan on putting spinning disks in it, and now I have something that was MUCH cheaper than something like the ARECA 4036, and it holds 16 disks instead of 8.

So to anyone looking to use this at home, just get the antec fans I mentioned above, as well as a 40x40x28 fan replacement for ths PS (3000 to 8000rpm range fans will be much quiter), and you will have a great home solution that is quiet and cheap.

I'm a new member here, but lurking for a while and learning and thank everything for the help.
 
Does anyone know if you can connect another one of these to the SAS out port? For example, do they daisy chain together?

*EDIT - They do.
 
Last edited:
Regarding the drives used for this chassis, is there any advantage to using WD 4tb sas drives vs WD RED (not pro) sata drives. I already have Four of the RED 4tb satas, so if they are reliable long term for this 16 drives chassis I will go that route. Any reason to need SAS or to need the WD pro that states they are for larger chassis? This is just a large media server for my house with the only uses being to storage and playback / stream Blu Ray rips, etc.
 
You must log in or register to reply here.
Back
Top