Data Recorder Project

B

ben chi(f4)

2[H]4U
Joined
Mar 4, 2008
Messages
2,339
~40,000 to spend on a data recorder and I'm looking to build my own (remember the airplane project?)

Well, the time has come that I need to peice this thing together. Now, what I need is ~500mb read/write speeds. What would be the perfect scenario would be that I have a box full of pci-e ssd HDs, but space is limited and am I really going to find a motherboard to support 4+ of these?

So, this is what I'm looking at. I'm planning on getting two to four of these raid 0 then dumping the data onto another storage platform. What do you guys think? I haven't been able to find anything else that comes with the size that I need and the speed that I want. Maybe going with this motherboard.
 
Last edited:
As an Amazon Associate, HardForum may earn from qualifying purchases.
Did you $4,000...not $40,000?

Also there are plenty of Mobos that will support 4x of those PCIe SSDs
 
No... I meant $40,000.

nitro, could you show me the motherboards?
 
Last edited:
Thanks Freak. I'll look into this and keep you up to date. I'm not off to find a rugged case and build my system.
 
Do you mind discussion what the project is for and how you came to define your requirements?
 
I would roll my own PCI-SSD solution. The prepackaged ones use a crappy raid chip.

Do this:
Supermicro SC216 series 24x 2.5" slots
24x SSD of the size of your choice. At 256 GB, that's 6 TB total. I would go with Vertex Turbo with 1.41 firmware
3x LSI 9260 8i
Supermicro X8DAH+-F has enough PCI-E slots (you can put a 8x card in a 16x slots)

The bulk of the cost is in SSD which is around 23k. This build would cost less than your budget.

And after you build this, you can software raid the three raid controllers together. They are rated at 1.7 GB/s. We want to see the 5.1GB/s benchmark!
 
Last edited:
$40k? Buy, not build. If you decide to build it anyway, don't use any consumer parts. Your board should be Supermicro, not Gigabyte. Those PCIe SSDs that you linked to are a bloody ripoff as well. They are literally 4 SSDs attached to an LSI RAID card (which they charge a premium for when you could do it yourself for less with the appropriate case). Do yourself a favour and buy a nice SAN or server instead of building it yourself.
 
Yea, 40K system down and ya gotta call the egg or rma/refurb crap. That's a bad way tp strectch 40K.
 
What underlying technology are you using to store the data coming in?

Ie: database, flat file system, etc?
 
Ok, so I just got an e-mail from the Photofastssd guys and the long and short:
photofast said:
Maybe they took it off because MLC NAND Flash has skyrocketed in price. 512GB is currently about $3039.
Click to expand...

I'm waiting on the price for the 1tb 1GB/s drives so we can get 2x of these in the case. That supermicro mobo looks good too, although I don't think I'm going with that case since I won't be using the SATA/SAS drives.

Pyrolisitcal said:
Do this:
Supermicro SC216 series 24x 2.5" slots
24x SSD of the size of your choice. At 256 GB, that's 6 TB total. I would go with Vertex Turbo with 1.41 firmware
3x LSI 9260 8i
Supermicro X8DAH+-F has enough PCI-E slots (you can put a 8x card in a 16x slots)

The bulk of the cost is in SSD which is around 23k. This build would cost less than your budget.

And after you build this, you can software raid the three raid controllers together. They are rated at 1.7 GB/s. We want to see the 5.1GB/s benchmark!
Click to expand...
This also sounds like a pretty good option... will keep it in mind.
 
ben chi(f4) said:
Ok, so I just got an e-mail from the Photofastssd guys and the long and short:


I'm waiting on the price for the 1tb 1GB/s drives so we can get 2x of these in the case. That supermicro mobo looks good too, although I don't think I'm going with that case since I won't be using the SATA/SAS drives.


This also sounds like a pretty good option... will keep it in mind.
Click to expand...

If you need more write speed, you can trade it for space.

The Vertex EX are SLC with higher write speed but less space
 
Photofastssd(e-mail) said:
Checking.

Please realize that the ATTO benchmark shows these to be about 500MB/S read and write which is 1/2 the claim which is confirmed with the CrystalMark benchmark. I think that benchmark is relying on the cache.

If you want higher speeds, we recommend buying (4) V5 SSDs. The PCI-E G-Monster has (4) of the V3 (earlier slower models) mounted to a Promise RAID Card. This PCI-E G-Monster SSD is bootable.
Click to expand...


I think I'm going to go with Pyrolistical's build.

Pyro, how fast do you think the write speed will be?
 
With Vertex EX its about 200 MB/s seq write per drive. Each 9260 8i has a max throughput of 1.7 GB/s, so with 8 drives you can hope for 1.6 GB/s. But it'll definitely be less, but I would guess definitely more than 1 GB/s. So with three raid cards for 24 drives you'll have around 3 GB/s seq write
 
Sweet. I'm going to write this up and give it to the lab manager and see what he things. I wish i could take pictures for you guys.. but when I get it, I'll post scores for you.
 
Just priced out the machine to $36,024

RAM 18x 4GB DDR3 1333 = $2295.00
Supermicro x8dah+f = $487.00
LSI MegaRAID SAS 9260-8i 3x $480.00 = $1440.00
Supermicro SC216A-R900UB = $826.00
Photofast 2.5 256GB V5 SATA SSD (Rated Performance 270MB/S read, 270MB/S write, 300MB/S host interface ) 24x 256GB (We might not get all of these and just get enough for 2TB, then I'd just get two controllers) @$1,249.00 = $29,976.00
 
Just took a peak at the insides of the Photofast V5 and its very interesting. They are doing internal raid 0 on two Indilinx controllers, which is why they have such mad speed. Very cool they fit it in a 2.5" form factor. OCZ is doing a 2/4 controller configuration in 3.5" drives.
 
*update*
PhotofastSSD said:
Oh. I forgot to mention, Photofast HAS designed a 512GB V5! Same performance. If you want those, let me know and I can ask for the lead time. Obviously, that is a TON of flash, so I'll have to check and see if your quantity is possible. It is a slightly lower cost per GB than 256GB.

Pricing is per SSD, regardless of quantity as these are niche, build-to-order SSDs and don't cost Photofast any less to build in sub 100 quantities.

V5 256GB: $1299
V5 512GB: $1999
Click to expand...

I wish the 8GB DDR3 sticks weren't so expensive, I would of loved to go with max ram on this one... just to say we have it? Why not?
 
Ok, so all my parts are ordered...

Now..... which OS to go with :eek:

Either a *nix distro or Win...
 
I guess you've already ordered, but a Sun Fire x4540 would easily sustain >1GB/sec read/writes, and starts at $30K for 24TB (48x500GB). I'd imagine that you'd really quickly hit an I/O ceiling with too many SSD drives.
 
ikari,

Thanks for the reference. Although that is a nice case and yes we did order already, 4u is just way too big to fit in the vehicles that we're going to be mounting our data recorder into. My only shame is that I won't be able to take pictures of the build for you guys. :eek:
 
ben chi(f4) said:
ikari,

Thanks for the reference. Although that is a nice case and yes we did order already, 4u is just way too big to fit in the vehicles that we're going to be mounting our data recorder into. My only shame is that I won't be able to take pictures of the build for you guys. :eek:
Click to expand...
Vehicles, eh? DARPA? But yeah, you definitely want SSDs instead of spinning disks, then :D
 
Ok, so I finally got most of the hardware for my machine. I'm pretty upset at the fact that purchasing bought the wrong RAM. I ordered 18 4gb unbuffered ram and they bought me 36 2x4gb Registered ram. UGH! Anyway, I popped in my raid controllers and hooked everything up. Here's how I have my raid set up:
Code: 
Init State:         No Init
Stripe Size:      128KB
Read Policy:    Adaptive Read Head
Write Policy:    WB W/BBU
I/O Policy:        Cached I/O
Access Policy: RW
Disk Cache:     Unchanged

Then I used Windows 7 RAID capabilities to build a software based RAID0 using two RAID0 Volumes.

Here's my HD tune scores
Min 435
Max 626
Average 574

Access 1.3 ms
Burst 109.0

It seems as though these numbers although pretty decent seem to be pretty low. What could I do to tweek my system to push out the maximum numbers?
 
Ok, So I did some tweeking to my settings:

Code: 
Stripe Size: 1M
Read Policy: Adaptive Read Ahead
Write Policy: Write Thru
IO Policy: Direct IO
Disk Cache Policy: Enabled

Now HD tune scores are:
Min: 430mb/s
Max: 910mb/s
 
Pyro,

Gonna try but I'm not too familiar with the software... will get back to you when I figure it out.
 
ben chi(f4) said:
Is it really? We're easily pushing transfer rate 2Gbps.
Click to expand...
Random or sequential data? Do you really need super fast random access? 2 Gbps is doable with regular drives, maybe add some SSD cache just in case. For USD 40k buy something with support, instead of rolling your own. I know that the latter is more fun.
 
Why are you using windows software RAID?
That is whats killing you.

I have three Vertex's and I set them up in software RAID, and then on the Intel RAID and its twice as fast using the ICH10r on my mobo vs Windows 7 software raid.
 
I would break the software raid0 and run the benchmark and see what numbers you are getting.
 
Broke the software raid but I can't seem to get headturning speeds or any major improvement.

Ok, keeping the stripe size at 1m I'm getting

Min: 646mb/s
Max: 1054mb/s

I feel as though I'm missing something.
 
How many drives in the RAID set?
RAID0 across all the drives?
 
You must log in or register to reply here.
Back
Top