Can someone explain Microsoft Storage Pools and when to use them

X

x509

2[H]4U
Joined
Sep 20, 2009
Messages
2,630
Like the title says. I need to go beyond the official M$ blarney about how great they are. And what are the good use cases, and the not-good use cases.

I have all these old Hitachi 1 TB and 2 TB drives that are in great shape because they were used for backup, 1 drive for each year of backup files. I don't need to keep those older backup files.

In another thread someone recommended a Drobo unit, but they are very pricey.
 
I've not used it, but aside from being "free", there are (very likely) better third party options.

The real question is what exactly are you trying to accomplish? ie: what is your goal.
 
MS Storages Spaces works. I'd just set up mirrors of the largest pairs of drives you have and run with it.
 
Mirror setups work fine. Parity setups on storage spaces are terrible. I used it on two different computers and both times write speeds around down around 30-40MB/s. Obviously parity takes a write hit, but I have never seen it THAT bad before. Mirror setup works fine though.
I would also use ReFS instead of NTFS as the newer file system is significantly better.
 
Krispy Kritter said:
I've not used it, but aside from being "free", there are (very likely) better third party options.

The real question is what exactly are you trying to accomplish? ie: what is your goal.
Click to expand...
I would like some way to use the space in all these drives but with just one drive letter, and without spending a lot of $$$. I have a Corsair case with 3 unused drive bays, plus 2 unused front bays for 5 1/4" devices.

I just mentioned Storage Spaces because I saw a reference to that feature. But a good third-party option is always welcomed.
 
The basic features are 'free' but if you want to access any of the more advanced setups you have to have a datacenter OS license. They aren't available on standard.
 
Biznatch said:
The basic features are 'free' but if you want to access any of the more advanced setups you have to have a datacenter OS license. They aren't available on standard.
Click to expand...
My systems are Win 10 64 Pro
 
Jandor said:
Just use the raid feature of your hardware if there is one.
Click to expand...

Really, don't. Storage Spaces drives are portable between OS installations, like most modern RAID filesystems. Using the stuff built into consumer hardware isn't any faster and is far less portable. Much higher chance of losing data to something stupid.

Not that you shouldn't have backups, because you should.
 
IdiotInCharge said:
Really, don't. Storage Spaces drives are portable between OS installations, like most modern RAID filesystems. Using the stuff built into consumer hardware isn't any faster and is far less portable. Much higher chance of losing data to something stupid.

Not that you shouldn't have backups, because you should.
Click to expand...

IdiotInCharge. Good points, and now I have at least one reason to use Storage Spaces instead of just some random motherboard maker's RAID.
 
x509 said:
IdiotInCharge. Good points, and now I have at least one reason to use Storage Spaces instead of just some random motherboard maker's RAID.
Click to expand...

I've run it on 10 Pro and Server 2016- if that fits your use case, it's production-ready (there are enterprise SANs using it now).

On alternative I used for a while was to pass drives to a hypervisor and run an OS with ZFS. FreeBSD works well this way but there are many others. More CPU usage and significantly more memory usage going this route if you're looking for a high-performance solution for say databases, but it does work and there is no performance loss on the storage side at least with spinners or SATA SSDs. NVMe might be a different story.
 
I personally use Storage Spaces for tiered storage. You can use SSD and HDD, or NVMe + SSD (SATA) + HDD. It's possible to mix this with columns, different interleaves, etc., I also allocate a write cache to handle small writes. I also use plain Windows for RAID, if your intention is to use SS for that instead (both are portable) then the differences generally are CPU overhead and higher QD performance (SS with both) and SS is more flexible if you know PowerShell (on Windows 10). On another machine I use DrivePool instead but the use case is different: there, I have a MLC SSD for caching and a bunch of mismatched HDDs in a pool.
 
Maxx said:
I personally use Storage Spaces for tiered storage. You can use SSD and HDD, or NVMe + SSD (SATA) + HDD. It's possible to mix this with columns, different interleaves, etc., I also allocate a write cache to handle small writes. I also use plain Windows for RAID, if your intention is to use SS for that instead (both are portable) then the differences generally are CPU overhead and higher QD performance (SS with both) and SS is more flexible if you know PowerShell (on Windows 10). On another machine I use DrivePool instead but the use case is different: there, I have a MLC SSD for caching and a bunch of mismatched HDDs in a pool.
Click to expand...
OK. I'm the OP, and you completely lost me here. That's why I posted my original message. (Not meant to critisize, I'm the one who doesn't understand.)

Maybe I should have said that I'm not running an IT shop, but I do have a home LAN. Also, I don't know Powershell, just CMD.
 
No problem, I wasn't 100% sure how much detail you wanted (I can end up writing a lot if you let me).

Storage Spaces is a part of Windows 10 and Server that's somewhat similar to AMD's StoreMI (which is based on FuzeDrive). It lets you do software RAIDs, like stripes, mirrors, parity, etc., but also tiered storage. Tiered storage breaks your drives down into tiers - NVMe on top, then (SATA) SSD, then HDDs, although you can configure the specifics. Windows has a heatmap that basically determines which data is most important (most-accessed) and moves that to the faster tier(s). Additionally, it reserves some space on the higher tiers as a write cache to absorb smaller writes (e.g. SSDs are way faster at small writes than HDDs). This process can be scheduled and the overall procedure means you don't have to worry about fragmentation.

Storage Spaces is portable in the sense you can move it to other installs since the SS information is stored on the drives. Windows 10 (as opposed to server) does not have a full-featured UI for SS, at least the last time I checked, which means the more complex setups require PowerShell. PowerShell is somewhat similar to CMD. If you know what you want to do, it's not actually that complicated, but I digress.

If you're not doing tiered storage then other options are using your board's controller, for example Intel's RST, which despite having its own ROM is still technically software RAID but less portable. Windows also can RAID directly in a management console (Disk Management under Computer Management) but it's not as flexible. SS is the most flexible. Performance-wise, SS tends to have more CPU overhead (usually negligible with the CPUs we have today) but also has better high queue depth/IOPS performance - this just means it's better at heavy server tasks. But this is a generalization.

DrivePool is software for Windows you can buy which just does pools. SS also does pools (in fact that is its main way of interfacing with drives) but DrivePool is more oriented at load-balancing and deduplication with tons of ways to configure that (add-ons). One add-on is the SSD Optimizer which essentially uses one or more SSDs as a write cache. For example if you were running Usenet, Torrent, Emby, Plex, etc. servers on a machine, incoming data would go to the SSD first and be unpacked and arranged and then later quickly (and evenly) dispersed to any number of HDDs. The advantage of pools is that you can add or remove drives more or less at will depending on the configuration, it's quite flexible.

There's other software that can do caching including with your system memory (RAM), like PrimoCache, which is a different beast since it uses a different type of caching (block). With RAIDs you typically use stripe for performance, mirror for redundancy, parity for similar, etc., not sure how much you know about that - it depends on what you're doing with the data.
 
Maxx said:
I personally use Storage Spaces for tiered storage. You can use SSD and HDD, or NVMe + SSD (SATA) + HDD. It's possible to mix this with columns, different interleaves, etc., I also allocate a write cache to handle small writes.
Click to expand...
Are you doing anything special to have different interleaves and columns on the different tiers? Every time I tried that it just failed to make the pool.
 
EniGmA1987 said:
Are you doing anything special to have different interleaves and columns on the different tiers? Every time I tried that it just failed to make the pool.
Click to expand...

My current setup is 2x120GB SSDs with 2x1TB HDDs. It's a bit of an unusual setup because the SSDs are in external enclosures. I actually have these in a two-column setup so it runs as a stripe while still being tiered, so it's quite possible to get upwards of 1GB/s off it. I say unusual because the SSDs are identical and use the same enclosure (the HDDs are also the same) so I have to name them specifically/manually for the commands to work right, also I have to specify the type for this reason (e.g. SSD or HDD).

Anyway, you can find my base setup (PowerShell commands) here. The storage tier sizes are set manually because you cannot use the maximum variable with this configuration (it'll error out) - there's some slack space needed for the heatmap and write cache (default 1GB I believe, which can be changed). There's a way to calculate that or you can just trial-and-error it. It's possible to manipulate the interleave as well (default is 256KB).

I don't have different interleaves/columns on different tiers, though. I experimented with it quite a bit and you can set interleave (and columns) with the new-storagetier lines, if that's what you mean.
 
Maxx said:
No problem, I wasn't 100% sure how much detail you wanted (I can end up writing a lot if you let me).

Storage Spaces is a part of Windows 10 and Server that's somewhat similar to AMD's StoreMI (which is based on FuzeDrive). It lets you do software RAIDs, like stripes, mirrors, parity, etc., but also tiered storage. Tiered storage breaks your drives down into tiers - NVMe on top, then (SATA) SSD, then HDDs, although you can configure the specifics. Windows has a heatmap that basically determines which data is most important (most-accessed) and moves that to the faster tier(s). Additionally, it reserves some space on the higher tiers as a write cache to absorb smaller writes (e.g. SSDs are way faster at small writes than HDDs). This process can be scheduled and the overall procedure means you don't have to worry about fragmentation.

Storage Spaces is portable in the sense you can move it to other installs since the SS information is stored on the drives. Windows 10 (as opposed to server) does not have a full-featured UI for SS, at least the last time I checked, which means the more complex setups require PowerShell. PowerShell is somewhat similar to CMD. If you know what you want to do, it's not actually that complicated, but I digress.

If you're not doing tiered storage then other options are using your board's controller, for example Intel's RST, which despite having its own ROM is still technically software RAID but less portable. Windows also can RAID directly in a management console (Disk Management under Computer Management) but it's not as flexible. SS is the most flexible. Performance-wise, SS tends to have more CPU overhead (usually negligible with the CPUs we have today) but also has better high queue depth/IOPS performance - this just means it's better at heavy server tasks. But this is a generalization.

DrivePool is software for Windows you can buy which just does pools. SS also does pools (in fact that is its main way of interfacing with drives) but DrivePool is more oriented at load-balancing and deduplication with tons of ways to configure that (add-ons). One add-on is the SSD Optimizer which essentially uses one or more SSDs as a write cache. For example if you were running Usenet, Torrent, Emby, Plex, etc. servers on a machine, incoming data would go to the SSD first and be unpacked and arranged and then later quickly (and evenly) dispersed to any number of HDDs. The advantage of pools is that you can add or remove drives more or less at will depending on the configuration, it's quite flexible.

There's other software that can do caching including with your system memory (RAM), like PrimoCache, which is a different beast since it uses a different type of caching (block). With RAIDs you typically use stripe for performance, mirror for redundancy, parity for similar, etc., not sure how much you know about that - it depends on what you're doing with the data.
Click to expand...
Thanks for the explanation. Now I have a good basis for deciding how/when/if I will use Storage Spaces.

I LIKEd your post.(y)
 
You must log in or register to reply here.
Back
Top