SSD Pool recommendation for VM datastore?

[farscapesg1]

Currently I'm running all my VMs at home on a striped-mirror of 4 750GB Seagate SATA drives (7200 RPM) using ZFS on OpenIndiana. I really want to replace the datastore with one housed on SSD drives and I'm looking for opinions

I'm currently running around 15 VMs, and it would all fit easily into 512GB as I'm currently only using about 300GB, and some of that can be put on slower storage (I have a RAIDZ2 Pool of 8 2TB drives with an NFS datastore shared). While the environment is used for lab purposes, it also houses some VMs used by the entire family (Untangle, SageTV server, some VMs my wife uses to print coupons, WHS),

Right now I'm looking at either doing a mirror of 2 512GB drives, or a RAIDZ set of 3 256GB drives. Obviously, either option will provide noticeable improvements in I/O and data rates (connecting to an M1015 controller). I have a feeling the "no brainer" answer is the RAIDZ pool of 256GB drives, but thought I would seek other opinions. Also keep in mind the ability to expand, though migrating the machines to slower storage and rebuilding the SSD pool would be pretty painless since it is a home environment.

Also, any recommendations on drives to do this? Samsung 840's (non-pro) good enough? Or try to pick up some of the older Crucial M4, Samsung 830's, or SanDisk?

 

[bobstone]

I would stick with Intel drives, I have read that some drives have a REALLLLLY low life span on esxi and while I can not find the list I do know that Intel drives where considered good for ESXi.

 

[spazoid]

Why would their life expectancy be lower as an ESXi datastore? ESXi wont even be handling the drives, from what OP says (raidz).

I would have no hesitations using an 840 non-pro for home use, and if (when) you can just replace it, and it'll very likely be cheaper in the long run.

 

[farscapesg1]

I have read that some drives have a REALLLLLY low life span on esxi and while I can not find the list I do know that Intel drives where considered good for ESXi.

Hmm, I'll have to do some searching on this. Using OpenIndiana (with Napp-it), the ZFS pools reserve 10% space. I'm thinking about bumping that up to 20%. For 512GB of total space, reserving 20% will still leave me with about 400GB usable, which would cover everything I'm currently running and give me a little extra room.

My understanding is that modern SSDs have garbage collection/trim/whatever enabled in the firmware now, so the OS managing the disks shouldn't matter correct?

 

[bobstone]

ok, still cant find the write-up but I just remembered it had to do with esxi Cache drives.

Some controllers on some SSD's caused them to die in less then 6 months when used on even a low volume esxi box.

The caching was making so many writes in so short of a time that, while it did not go past the drive life write expectancy, it still killed the drives right quick due to the controller being unable to handle it long term ( i.e. it was controller specific issue not a slc vs mlc issue ), with a data-store this should not be an issue so please disregard what I said.

 

[dilidolo]

Samsung non-pro is not good candidate for datastore. The pro version may be ok.

If you can afford, get Intel DC S3700 or Crucial p400m

 

[farscapesg1]

If you can afford, get Intel DC S3700 or Crucial p400m

Yeah... that ain't happening for a home lab environment $2/GB+ for the Intel? Don't think that is going to be in the budget at any time.

 

[dilidolo]

Yeah... that ain't happening for a home lab environment $2/GB+ for the Intel? Don't think that is going to be in the budget at any time.

Then I personally would use 3*256G 840 Pro in raidz

 

[Aesma]

Hmm, I'll have to do some searching on this. Using OpenIndiana (with Napp-it), the ZFS pools reserve 10% space. I'm thinking about bumping that up to 20%. For 512GB of total space, reserving 20% will still leave me with about 400GB usable, which would cover everything I'm currently running and give me a little extra room.

My understanding is that modern SSDs have garbage collection/trim/whatever enabled in the firmware now, so the OS managing the disks shouldn't matter correct?

I didn't know that ZFS did that, interesting.

Whatever is in the firmware, there is no way it can know what is data and what is garbage, the only way is if it receives the trim command from the file system.

 

[spazoid]

ok, still cant find the write-up but I just remembered it had to do with esxi Cache drives.

Some controllers on some SSD's caused them to die in less then 6 months when used on even a low volume esxi box.

The caching was making so many writes in so short of a time that, while it did not go past the drive life write expectancy, it still killed the drives right quick due to the controller being unable to handle it long term ( i.e. it was controller specific issue not a slc vs mlc issue ), with a data-store this should not be an issue so please disregard what I said.

But in this case it's not being used as an ESXi cache drive, it's being used as a datastore, and it's being managed by ZFS - completely different scenario..

Oh, and comments on using S3700's, ZeusRAM and other stuff like that - I don't think that's relevant for home use, unless we're talking people with too much money, and in that case, I don't think the OP would even bother asking these questions and just go out and buy some huge drives...

 

[Aesma]

Well apparently he has three fully fledged servers at home...

 

[spankit]

It should be noted that you would get the same IOPS performance from two identical drives in a MIRROR than you would using 3 of the same drives in a RAIDZ. The only thing that will change is the Sequential performance. Since you are using SSD's and this is in a home environment IOPS performance may be the last thing your worried about but that's not to say it should be ignored. Though from a pricing standpoint it may be less cost efficient, you would end up with better performance overall using 4x 120GB SSD's in two MIRROR'd vdevs.

Link: https://blogs.oracle.com/roch/entry/when_to_and_not_to

 

[farscapesg1]

Well apparently he has three fully fledged servers at home...

Haha.. yeah but they are all white box with most of the storage cobbled together from "freebies" at work. The rest was either new parts or F/S (with all the procs coming from F/S threads) collected over a couple years (started with one host as an all-in-one, added a second host, then broke out the storage VM into a physical box).

Still... enterprise equipment is a little out of my price range for this

 

[farscapesg1]

It should be noted that you would get the same IOPS performance from two identical drives in a MIRROR than you would using 3 of the same drives in a RAIDZ. The only thing that will change is the Sequential performance. Since you are using SSD's and this is in a home environment IOPS performance may be the last thing your worried about but that's not to say it should be ignored. Though from a pricing standpoint it may be less cost efficient, you would end up with better performance overall using 4x 120GB SSD's in two MIRROR'd vdevs.

Link: https://blogs.oracle.com/roch/entry/when_to_and_not_to

Yeah, I get that. That's why I use a pool of 4 750GB drives in two mirrored vdevs for my VMs right now, as opposed to just putting them on the RAIDZ2 array. The thing is, a single SSD will smoke what I have currently in IOPS anyways... so anything more than that is just extra icing

Though, with current prices I guess I can really get 2 x 256 drives for about the same price as a single 512, so I could go with 4 x 256 drives and then decide whether I want them in mirrored vdevs or a RAIDz....

I guess at this point the bigger concern is longevity of the drives running under OpenIndiana (or some other non-Windows OS) and hosting VMs. I'm not running any database intensive VMs, the most intensive from that point would be SCCM 2012 to provide patch management and software deployment to my home network. I do have one VM running as a bittorrent/usenet downloader, but I'm guessing the recommendation would be to put the temporary download locations on a vdisk on the RAIDZ2 datastore to minimize writes to the SSDs (completed files end up on that pool anyways).