50TB+ flash array/storage you recently purchased

M

mp3turbo

n00b
Joined
Dec 25, 2013
Messages
16
Guys,

pretty soon, we might start to look for "price CONSCIOUS" flash storage with approx. 60TB usable capacity right now with expansion to 90TB expected in a year. Will be used for data warehousing appz on 20 ESX v5.5 hosts, 10Gbit FCoE, single location, single room. Apologies for not mentioning detailed environment specification as I don't consider it to be necessary now [see below, please, before shooting me and don't talk about redundancy].

It does NOT have to have the best-of-the-world-all-bells-and-whistles, ie. we can survive without dedup. Basically I can't recall any really sub-par flash vendor as of features, so I'm not worried too much. Usually if a vendor X is missing this or that functionality, they will add it within 6 to 12 months which is extremely fine with us on general basis. Of course, careful consideration / comparison will be performed.

Before I run into every flash vendor in the world, I would like to hear your experience as of configuration/price. I'd be interested in info like "we procured ABC with xx TB raw capacity, performance specified as xxxx IOPS / yyyy MB/s for $$$$$$ and all that happened in October 2014". That will easily show me where to look and this is the pure reason why I'm not mentioning utter details of our environment.

Big thanks in advance. Feel free to post additional questions, but you generally see what I'm after. The list prices are useless and we can't deal straight away with 15+ suppliers as it's time consuming and we simply don't have the resources.

A note : we are not interested in any SSD caching, tiers etc in standard arrays as their architecture generally is not as capable as all-flash arrays have. Also we believe there is no relevant price & performance advantage for us, we checked with vendors of storages we already use - knocked my socks off when I've heard the price - so we'd rather buy entirely new flash storage for this DWH, effectively freeing up existing resources. This was cross-posted in virtualized computing forum as I don't know which is better.
 
Have you looked at anything yet? How about posting up what options you're considering you may get more feedback/reviews that way.
 
I know this topic is too complicated - we've already checked Kaminario, like the architecture, performance, don't know the price (sigh) and don't like what we consider weak density - at the time we checked, about 4 months ago, they only had 24 drives in 2U shelf. Compared to now-pretty-usual 72+ drives in 4U, this seems to be not so great.

We don't have any options on the table now, nothing has been investigated as of real pricing and business case. We just start to look around now and that is the main reason why I created this topic. If somebody says "we procured Pure with 76TB raw (96x 800GB SSDs) in 6U for $150.000 on Christmas with support contract for $15.000 a year", I easily know whom to call.
 
If you want us to do your homework, then just ask it without hiding it.

Cheap SSD is going for roughly $0.50 AUD per GB here down under. That means domestic desktop SSD, not enterprise that is more than twice to 4 times as much.

Now add in RAID cards, yes, you will need more than one if you want to use all the pipe. Otherwise, with only a 10GbE, a single high-end card used with good expanders will see the the pipe filled easily.

OS, your choice, doesn't matter until you know what the rest of the systems are, if native Windows, then it will be the Server 2012R2, you can figure the rest.

Chassis, SuperMicro is probably best solution off the shelf but there are plenty of niche vendors pushing some monster chassis's packing huge drive counts along with equally big price tags.
 
Benji, you are really lost : we are not going to build anything ourselves. I know what desktop-grade SSDs cost and what are their features, performance characteristics (consistency, latency, endurance etc). If I was about this home-grown thing, I would snap in as-many-as-I-can Sammies 845DC Pro (800GB version found for 640eur with VAT where I live) to some $5000 SuperMicro box as you say, hooked on several pci-e 3.0 8x LSI 93xx or Adaptec 71605/72405 adapters (400eur/piece) with several 56Gbit Mellanox ConnectX-3 cards (300US$) off ebay. And I'd achieve 2 million IOPS 4kB when done properly (700.000 IOPS per HBA, measured in real tests everywhere, storagereview.com, thessdreview.com, plus sequential throughput 18GB/s achieved on three 56Gbit cards by Jose Baretto with RDMA, Windows2012R2). The thing would cost much less than six figures.

I'd have two such boxes for redundancy purposes, letting them mirror synchronously. Dozens of options how to do that.

Would you put multi-million dollar mission critical data on such "storage", would you bet your job and your name ? Have you ever had personal audit of your premises by multiM$$$ customer, would you show off "hey look this is what I welded together yesterday, I don't have anybody to call when situation gets nasty - but sure you can rely on me, your data will be safe" ???


I just wanted to hear what people are satisfied with. In the shadow topic I created, there was a guy claiming [some] vendor pricematched competitor - going down from 1.4 million to 0.4 million. That's the information I'm looking for. Who inflates how much. Unfortunately the name was not said but I'm not in 1.4M league so it's not important for me. If everybody try to sell me something for 0.5M and someone for 1.5M, I really know what to think about it.

I'm trying to find PERSONAL experience what are current good offers. If somebody says "look at XYZ, we've been investigating storage for 4 months and they were the winners, followed by ABC, DEF vendors". Sure, I can run to every single vendor in the world and find this out myself - it will take me exactly the same four months.

Sharing experience and knowledge, advising and spreading the word is what discussion forums are about, right ?
 
LOL @ the long explanation of how you're NOT going to do it, and then to throw it in the persons face who was trying to help you.

Like I said before... maybe posting up some of your thoughts, about what you want/don't want, have looked at, etc... may help a bit more than you think ;)
 
You answered your own question. Tell 4 vendors like HNAS, Isilon, NetApp,and Nexenta your iops/throughput requirements and have them bid against each other. You say it's too time consuming but if you tell these vendors you're making your decision in a week and don't have time for presentations, they'll damn sure get their quotes to you quickly.
 
mp3turbo-
You have not informed any of us as to the most important details of what you seek... These would be the specific applications and services you are running which need this storage. How many users are you supporting? What are your needs as to IOPS, latency, replication, backup, DR etc? What have you tried for which standard (spinning) or hybrid (SS/spinning) storage elements failed to suffice? As common accounting principles age your depreciation of CAPEX at 5 years, how do you see your needs growing over the next 5 years and plan for them now.
 
mp3turbo said:
Cheap and pissy shot with gross claimed figures and plenty of knowledge gained from forums and paid review sites
Click to expand...

Well done on clearing up the question I was now asking, are you a wanker?

CopyRunStart has given you the response you need.
 
Make sure that you talk to Simplivity as well. They're doing some nifty stuff eliminating iops by deduping the initial writes before flash/mechanical storage. That translates well into replication and backup as well. Let me know if you need a rep's name.
 
You must log in or register to reply here.
Back
Top