Cheap ZFS server outperforms NetApp

If you follow the links in the article it spells out exactly the hardware:

Sun

NetApp

My quick take on it is the NetApp test was done in 2010, and used almost entirely disks in RAID-DP to hit the number (not much memory and no SSDs). The new Sun test has only 1/4 the capacity but hits the IOPS number by using a ton of memory and SSD cache on top of striped disks.

All in all I don't think it's really a worthy comparison. I'd like to see a test with similar usable space, sun doing RAID-Z2, and SSD cache in the NetApp.
 
It's easy to build something like that at home, it harder to keep it running and stable.

I don't know how the oracle pricing works, but you are paying for support. And normally with something that big, you really want that support, unless you don't mind downtime.
 
Another thing to note, the Oracle (Sun) solution didn't have any data protection at all. They are using RAID 0 which I think is just silly for an enterprise benchmark. From the spec page:

The pools are then set up to stripe the data (RAID0) across all 17 drives.
Click to expand...

At least the NetApp test was close to a possible real-world setup.
 
apnar said:
My quick take on it is the NetApp test was done in 2010, and used almost entirely disks in RAID-DP to hit the number (not much memory and no SSDs). The new Sun test has only 1/4 the capacity but hits the IOPS number by using a ton of memory and SSD cache on top of striped disks.

All in all I don't think it's really a worthy comparison. I'd like to see a test with similar usable space, sun doing RAID-Z2, and SSD cache in the NetApp.
Click to expand...

apnar said:
Another thing to note, the Oracle (Sun) solution didn't have any data protection at all. They are using RAID 0 which I think is just silly for an enterprise benchmark. From the spec page:

At least the NetApp test was close to a possible real-world setup.
Click to expand...
Agreed. Your points are both valid.

But, the ZFS server costed 180K USD. The NetApp server price was another million USD. Let us say that the ZFS server increased the price another 100K, then it would have as many disks as the NetApp server.

My point is: why is the price difference so huge? I mean, if NetApp increased the price another million bucks, NetApp could also use lot of RAM.

What do you think about the price difference?
 
brutalizer said:
What do you think about the price difference?
Click to expand...

I think you're underestimating the cost of enterprise SAS disks :)

I also think the price difference is real but not nearly as big as what it seems once you start evening out the configurations. It doesn't appear that the prices are listed as part of the test specs so it's hard to do an actual comparison. As the Reg mentioned the NetApp price is what Oracle says that system costs so I don't know how accurate it is either.

I couldn't find prices for the Oracle hardware, but I found this one for NetApp here (googled "netapp price list")

List prices:
2x FAS3270 = $42,000 x2 = $84,000
4x SAS Host Adapter X2065A-R6 = $2000 x4 = $8,000
15x DS4243-1511-24S-QS-R5 = $72,446 x15 = $1,086,690
2x X1139A-R6 (10 gig-e) = $4500 x2 = $9,000
2x NFS license (now included) = 0

total: $1,187,690

That doesn't include the test servers or network gear, but those aren't NetApp equipment. Looking at the total NetApp cost, 91% of it is just for disk drives and their shelves so you can't easily ignore the cost of the extra storage space.

Looking at the Oracle config, we'll assume that they needed to use RAID-0 to hit the IOPS number so turning it into usable space we'll keep the same striping but use RAID-1, right off the bat that doubles the disk count. Then to get us to the same exported capacity of the NetApp we need to multiply that by 3.3. So we'd be looking at adding another 5 times as many disk shelves to the Oracle solution for a total of an additional 30 shelves. As I mentioned I couldn't find a price for an Oracle J4410 with 24x300GB 15K disks, the comparable NetApp model (DS4243-1507-24S-QS-R5) sells for $58k, so lets be optimistic and assume Oracle is only half the cost around $30k a shelf. 30 shelves at $30k each is another 900,000 which would put it in same ball park of the NetApp for the same usable storage space.

Looking at it another way we could drop 10 shelves from the NetApp to bring it inline with the Oracle size and cut it's cost $725,000. You'd likely need to add in some SSD cache to get the IOPS back up though for another $40,000 (2x 256MB Flash Cache, X1937A-R5-C). It'd still be more expensive, but much closer in price.

Anyway, I don't think there is much to be gained in comparing the two entries as they show very different things. The NetApp one seems intended to show how well their controller scales as you add lots of spindles. The Oracle one was intended to show how you can use lots of memory and SSDs to hit an IOPS number without the spindles you'd historically need to get there. I reality a deployment would likely use both approaches in a balanced way to hit the size and IOPS requirements.

The one thing I get from these is that NetApp submitted a setup that someone may actually use in real life for it's benchmark (even included spare disks). Oracle submitted a setup that seems to only exist just to get the number for marketing reasons.
 
Ok, so you mean that if Oracle added as many disks as NetApp, the Oracle server would have costed another million bucks? Is this what you think, or do you know it is a fact? This sounds weird?

Regarding the config. I agree that Oracles server is configured for benchmarks, but assume Oracle reconfigures for real life and add more disks, I still doubt that Oracle would have to increase the price another million. I do believe that the Oracle price would still be much much lower.

I dont see the point why Oracle configured as Raid-0. I mean, Oracles server is so much cheaper, they could have added more disks and configured in raidz2 and still have a server much cheaper than NetApp.
 
I found some pricing for Oracle here, here is the break down which I think is pretty interesting:

2x 7320 (includes 48gig of mem & 2 read SSDs) = $93,718
48x 4 GB DDR3-1333 registered low-voltage DIMM = $118 x48 = $5,564
(I'm guessing the tested system used 8gig DIMMs which aren't on the site)
2x Sun PCI-E Dual 10GbE Fiber = $1,896 x2 = $3,792
4x 10Gbps Short Wave Pluggable Transceiver (SFP+) = $1,008 x4 = $4,032
8x 512GB Solid State Drive SATA-2 = $4,478 x4 (4 included above) = $17,912
8x SAS-2 73GB 3.5-inch SSD Write Flash Accelerator = $5,004 x8 = $40,032

Total without disks or shelves: $165,050

I still can't find the price for the J4410 disk shelf. The best I could do was find the price of adding a 300GB 15k RPM disk to another one of their storage appliances here. Each disk costs $623, so with 136 disks that's a total of $84,728 without the cost of the shelf.

So now the total is $249,778 without the shelves but including the disks. The Reg says Oracle claims the entire system only cost $179,602 so something doesn't add up. These are all list prices which we know no one actually pays but still the comparison should be made on a similar basis.

Anyway, your thought that the Oracle controller is so much cheaper isn't really that true. The NetApp without disks is $101k but the Oracle controller without disks or SSDs is $107k.
 
apnar said:
I found some pricing for Oracle

So now the total is $249,778 without the shelves but including the disks. The Reg says Oracle claims the entire system only cost $179,602 so something doesn't add up.
Click to expand...
I think it adds up perfectly well. It is well known that if you buy spare parts indivdually, everything gets more expensive. If you buy everything as a package, it is cheaper.

Anyway, your thought that the Oracle controller is so much cheaper isn't really that true. The NetApp without disks is $101k but the Oracle controller without disks or SSDs is $107k.
Click to expand...
Ok, so how do you explain the 1 million USD price difference? Maybe the hardware is similar, which means they should have similar prices. But, NetApp adds another million on top. Why not two millions ontop? Or three?
 
brutalizer said:
Ok, so how do you explain the 1 million USD price difference? Maybe the hardware is similar, which means they should have similar prices. But, NetApp adds another million on top. Why not two millions ontop? Or three?
Click to expand...

15x DS4243-1511-24S-QS-R5 = $72,446 x15 = $1,086,690

You spent over a million on shelves and disk with the netapp.
 
ND40oz said:
15x DS4243-1511-24S-QS-R5 = $72,446 x15 = $1,086,690

You spent over a million on shelves and disk with the netapp.
Click to expand...
I dont understand this. The Oracle ZFS server has half the number of disks, and still Oracle charges 200K, not a million.

Can you explain why?
 
brutalizer said:
I dont understand this. The Oracle ZFS server has half the number of disks, and still Oracle charges 200K, not a million.

Can you explain why?
Click to expand...

You're purchasing 360 - 450GB 15k SAS drives plus another 15 trays to hold them and all the cabling. Plus their pricing is from Sept. 2010, it's over 1.5 years old at this point, don't you think it's changed in that time span?

I believe the 3270s come with flashcache by default now as well.
 
Ok you mean the big price difference of 1 million is explained by
-double the number of disks
-prices are old

In that case, I hope NetApp will update to the new low prices. You are implying their prices have been lowered ("dont you think its changed in that time span?"). I wonder how much the NetApp server costs now. At least half the price, I hope. If it costs half the price, it it still 600K. NetApp sports twice the number of disks, so it should cost twice the price of the Oracle server. Thus, the NetApp server should cost 400K or so.

I suggest you email NetApp and tell them that their prices are 1.5 years old and outdated, and that the website should be updated to reflect the new prices that are much lower.
 
Why would I need to email them, I have their current prices for the products we buy? If you need pricing, call your reseller and ask them for quotes. But don't go off of a Register article where they last priced and test a unit a year and a half ago.
 
DUDE!! I am going to recommend to all my clients that they should totally build RAID 0 ZFS boxes to replace their EMC and Netapp SANs. OMFG what a ripoff!!!!!
 
brutalizer said:
Oracle ZFS server cost 180 K USD and is 32% faster than NetApp which costs more than ten times more (1.200 K USD).
http://www.theregister.co.uk/2012/03/15/oracle_zfs_spec_result/

Maybe you could mimick the ZFS server and build a cheap home version? How is it configured? What hardware?
Click to expand...

I love ZFS and that's what I use at home for my lab, it has 80% basic features of what NetApp has.

However, NetApp provides complete solutions such as data protection, dr, etc. also better certification with different app vendors. Flash Cache in NetApp is not the best, but works, and it does change the game.

If you just need basic storage, ZFS will work very well. But if you want complete storage solution, I wouldn't use current ZFS products.
 
Child of Wonder said:
RAID 0 is what Oracle used in their test. What's good enough for Oracle is good enough for me! ;)
Click to expand...

Maybe you should read the test.

They (Oracle) used RAID1 on all of the storage disks (300gb 15k spinners), where the $1.6M NetApp uses 17-disk RAID6 pools (450gb 15k spinners)... considerably better parity on the Oracle box.

The NetApp has 25% more (usable) storage, but also costs 4x as much with worse parity.
 
brutalizer said:
Ok, here is a new benchmark of an Oracle ZFS server vs NetApp.
https://blogs.oracle.com/si/entry/7420_spec_sfs_torches_netapp
It would be cool to build a similar ZFS server at home. :)

ZFS gives more IOPS, for a price of $430K. NetApp costs $1600K. The number of disks are roughly the same. How can you explain this big price difference?
Click to expand...

I'm not going to dive into the in-depth analysis like I did above, but at first glance I would say it all depends what you are comparing. They are talking $/IOPS, but look at $/exported TB. The Oracle has 280 disks yet manages to only export 36TB, yielding $11.8k/TB. The NetApp 3270 has 360 disks (not too much more then Oracle) but manages to export triple the space at 110TB, so $9.9k/TB. If price per TB is of interest the NetApp 3270 wins on the face.

At least this time they are using RAID1 instead of just RAID0. They are also comparing hardware they released last month to NetApp hardware a year and a half old.

Overall, it's just Oracle showing how you can now use memory and SSDs to hit higher IOPS at lower prices. Of course you could also add Flash Cache to NetApps as well.
 
Last edited:
Child of Wonder said:
RAID 0 is what Oracle used in their test. What's good enough for Oracle is good enough for me! ;)
Click to expand...
No, Oracle did not use RAID 0. Oracle used mirroring. There are two different Oracle benchmarks. This is a new benchmark, and another one.



apnar said:
I'm not going to dive into the in-depth analysis like I did above, but at first glance I would say it all depends what you are comparing. They are talking $/IOPS, but look at $/exported TB. The Oracle has 280 disks yet manages to only export 36TB, yielding $11.8k/TB. The NetApp 3270 has 360 disks (not too much more then Oracle) but manages to export triple the space at 110TB, so $9.9k/TB. If price per TB is of interest the NetApp 3270 wins on the face.
Click to expand...
NetApp uses 450GB disks. Oracle uses 300GB disks.

Overall, it's just Oracle showing how you can now use memory and SSDs to hit higher IOPS at lower prices.
Click to expand...
Yes, I think this is the geist of the Oracle benchmark. Oracle wants to show that they have lower prices. Not 20-25% lower prices, but much lower. You can buy several Oracle servers for the price of one slower NetApp server. Oracle succeeded.

NetApp, EMC, etc has been mocking Sun for not showing any ZFS benchmarks. Sun's answer was that those benches are obsolete and not interesting. However, Oracle has no such concerns, so Oracle shows ZFS benchmarks much to the chagrin of other storage companies. ZFS is a game changer, now anyone can build a cheap safe solution at home anyone can afford. The Oracle ZFS server is basically a PC with lot of disks, running Solaris. No need for proprietary software and special hardware to compete anymore. :)
 
I'm not going to dive into the in-depth analysis like I did above, but at first glance I would say it all depends what you are comparing. They are talking $/IOPS, but look at $/exported TB. The Oracle has 280 disks yet manages to only export 36TB, yielding $11.8k/TB. The NetApp 3270 has 360 disks (not too much more then Oracle) but manages to export triple the space at 110TB, so $9.9k/TB. If price per TB is of interest the NetApp 3270 wins on the face.
Click to expand...
This analysis doesn't really make sense, since both Oracle and Netapp are absurdly overpriced on the basis of $/TB. Anyone who buys a SAN cluster is presumably provisioning for IOPS, not for disk space.
 
Last edited:
brutalizer said:
No, Oracle did not use RAID 0. Oracle used mirroring. There are two different Oracle benchmarks. This is a new benchmark, and another one.
Click to expand...

I was referring to the initial post.

From the link in the opening post:

The storage configuration consists of 6 shelves, 4 with 24 disk drives, 2 with 20 disk drives and 4 write flash devices. Each controller head has 4 read flash accelerators. Each controller is configured to use 68 disk drives, 4 write flash accelerator devices and 4 read flash accelerator devices. The controller is then set up with pools by dividing the disk drives, write flash accelerators and read flash accelerators into 4 pools. Each of the controller's pools is configured with 17 disk drives, 1 write flash accelerator and 1 read flash accelerator. The pools are then set up to stripe the data (RAID0) across all 17 drives. The write flash accelerator in each pool is used for the ZFS Intent Log (ZIL) and the read flash accelerator is used as a level 2 cache (L2ARC) for the pool. All pools are configured with 4 ZFS filesystems each. Since each controller is configured with 4 pools and each pool contains 4 ZFS filesystem, in total each controller has 16 ZFS filesystems. The SUT has 32 ZFS filesystems for the benchmark.
Click to expand...

In that test they did use RAID 0. How else would you get 37.1TB of usable space from 136x300GB drives?
 
thefreeaccount said:
This analysis doesn't really make sense, since both Oracle and Netapp are absurdly overpriced on the basis of $/TB. Anyone who buys a SAN cluster is presumably provisioning for IOPS, not for disk space.
Click to expand...

I didn't mean to infer that someone would buy solely on the basis of $/TB. Only that it's one of the many considerations. Generally when purchasing you have a set of requirements, you need X TB of space that can service Y IOPS and has features A, B, and C. If for your project X is 36 and Y is 260k then the Oracle solution is likely cheaper. If X happens to be 110 and Y is only 100k then the NetApp solution would be cheaper. Even then it's often the A, B, and C features that push you one way or the other. Things like HA, snapshots, support, DR, expandability, etc.

I also seriously question the prices. I'd like to see what a real quote looks like to a large enterprise from each vendor. I haven't purchased any storage from Sun since they became Oracle but I'm guessing their retail prices are much closer to what you'd actually pay then NetApps are. Not that there is anything wrong with that, I even prefer it, I just know that if I went to my NetApp rep and asked for the 3270 config in the benchmark my price would be much, much, much lower then what's shown there.
 
You must log in or register to reply here.
Back
Top