Sandforce 1200 write amplification and free space

[Decibel]

Anyone out there have any experience with the Sandforce controllers?

My Indilinx based Patriot Torqx died a few weeks back. (They had a bad batch go out that has been causing them a lot of grief.) As a replacement they shipped me a new Sandforce based Inferno.

I've been doing a bit of reading about how the Sandforce controllers work. From some articles at Anand it seems that they compress the data and don't write as much to the drive. One of the examples listed is that for a 25gb Windows and Office install only 11gb is written to the memory.

My question - Say that 11gb is written for 25gb to a 100gb drive, does Windows see the drive with 89gb or 75gb free.

I could answer my own question in a few hours, but I'm debating keeping the drive or putting it up in the For Sale/Trade Forum. I don't want to take the shrink wrap off if I'm going to sell it.

We had a really wet spring here in California. The water is huge... Don't get me wrong, I love the SSD performance, but I really, really want a new whitewater kayak.

 

[FrostBite]

My question - Say that 11gb is written for 25gb to a 100gb drive, does Windows see the drive with 89gb or 75gb free.

Windows will see 25GB written so 75GB free if it has 100GB available. Write amplication ahs to do with how much flash memory is written and their life span. Intel has stated 1.1x so if you write a 10GB file, it will still show up as 10GB used, but it will actually write 11GB on the flash drive. The write amplication for Sandforce use some sort of compression for writing onto the flash that is less than 1, it has nothing to do with space, but with just how fast flash memory is writen/erased which reduces its life span.

As for the Sandforce drives themselves, I would definately keep it. It has benched quite well on various websites, and really, almost any SSD with a decent controller these days will be an absolutely huge jump from a traditionally HHD.

 

[Decibel]

That makes complete sense, I must be thinking about the issue to much.

- Because -

Here's the specific quote from Anand's Vertex 2 Pro article that's catching me up:

SandForce states that a full install of Windows 7 + Office 2007 results in 25GB of writes to the host, yet only 11GB of writes are passed on to the drive. In other words, 25GBs of files are written and available on the SSD, but only 11GB of flash is actually occupied. Clearly it’s not bit-for-bit data storage.

They already have something like 28gb set aside as spare area. If I were fill the 100gb drive with data that can hit the claimed 0.5x compression, on that "full" drive 78gb of the flash would be unused.

That just seems off. (128gb physical flash, 28gb reserved, 50gb thrown away...)

I understand write amplification going into the greater than 1x range. It's the functionality of how it works with less than 1x that I'm still not quite understanding. The flash is not occupied, why can't I use it?

Is the OS or hardware just not smart enough for it to be available?

And I still really want a new kayak.

 

[FrostBite]

They already have something like 28gb set aside as spare area. If I were fill the 100gb drive with data that can hit the claimed 0.5x compression, on that "full" drive 78gb of the flash would be unused.

That just seems off. (128gb physical flash, 28gb reserved, 50gb thrown away...)

You are going at it wrong. Write amplifaction is not for compression, and what Sandforce is focusing upon is writing on the flash memory as little as possible. The reasoning behind this is that right now, only the best (and expensive) flash memory can be used for SSD's because their write life is roughly 10k write cycles (compared to thumb drives at 4k-5k). What Sandforce wants to do is use the cheaper flash memory (4k-5k write cycle), but just write on the flash memory less hence you still get the same lifespan; Sandforce has this information and a ton more on their website. So, the gist of it is that they will compress/encode/encrypt the data so that the flash memory is used in a much lighter fashion.

I understand write amplification going into the greater than 1x range. It's the functionality of how it works with less than 1x that I'm still not quite understanding. The flash is not occupied, why can't I use it?

Write amplication for the Sandforce is not always .5x, sometimes it can be nearly 1x if the file is already compressed (jpegs, mp3, zip/rar) so there really isn't much to compress and therefore it has to write most of these types of files onto the flash module. Imagine if it was the other way around, you've got 100GB SSD and you have a write amplication of 10x (10MB = 100MB write), if you put 10GB on the SSD, does that mean it is full? No, you still have 90GB free, it's just that the controller has written on all the flash memory and therefore they start erasing/rewriting modules. If you were to write 100GB on a 100GB SSD with 10x write amplication, the SSD will literally write 1,000GB which means it will had to burn through quite a few erases/rewrites. Thankfully, there is a reserve area of 7% (up to 26%) which will help, but you'll see a LOT of stuttering and slowness while the controller tries to erase/rewrite hence so many problems from older controllers.

Is the OS or hardware just not smart enough for it to be available?

It's the controller, but the OS plays a role in such things as TRIM (or the lack of it, i.e. WinXP). Some earlier controllers had problems when the SSD was full, and had to erase/rewrite on the fly and there were massive stuttering of files. Exceptions are just extremely well built controllers like the Intel G1's which do not work with TRIM but are well designed. New and better controllers are coming out, but I think only Sandforce so far has tackled the issue in this particular manner.

 

[InvisiBill]

Here's the specific quote from Anand's Vertex 2 Pro article that's catching me up:

SandForce states that a full install of Windows 7 + Office 2007 results in 25GB of writes to the host, yet only 11GB of writes are passed on to the drive. In other words, 25GBs of files are written and available on the SSD, but only 11GB of flash is actually occupied. Clearly it’s not bit-for-bit data storage.

I agree, that seems a bit odd. I understand the concept of write amplification and that writing 10GB of actual data to the SSD might require 12GB worth of writes, for example. I also understand that compression could allow you to save 10GB of data in only 8GB of disk space.

However, if you write 25GB of files to the SSD, but only 11GB of the flash is actually used to hold that data, then you've got 14GB of extra flash that isn't being used. Perhaps it's similar NTFS compression, where you could write 25GB to your 100GB drive and it would show that it's taking up 11GB of disk space, leaving you with the 89GB free still (which could actually hold more than 89GB also, due to compressing the data to be put into that free space). You might be able to fit up to 200GB on your 100GB SSD, based on how compressible the data is.

Another option is that they could use that extra 14GB as spare area. The compression would also be giving the drive more "temp" flash to work with, which should also increase performance.

Something has to be happening with that extra 14GB of flash though. Does the drive compress the data solely for the benefits of writing less data, but still mark the full 25GB as being used? That seems quite wasteful. If they're not making the additional space available to the user, it seems like they should at least be using it as spare area (which could inherently be taking place, if the area is marked as clean and the controller is able to dynamically adjust to free space like that).

 

[FrostBite]

Something has to be happening with that extra 14GB of flash though. Does the drive compress the data solely for the benefits of writing less data?

Yup.

http://www.anandtech.com/show/2899/3
The whole write-less philosophy has tremendous implications for SSD performance. The less you write, the less you have to worry about garbage collection/cleaning and the less you have to worry about write amplification. This is how the SF controllers get by without having any external DRAM, there’s just no need. There are fairly large buffers on chip though, most likely on the order of a couple of MBs (more on this later).

http://hothardware.com/Articles/OCZ-Vertex-2-Pro-SSD-Powered-by-Sandforce/?page=2
This read-modify-write scenario takes its toll on the write endurance of the average SSD reducing its functional life dramatically. Typical write amplification for legacy SSDs has been in the neighborhood of up to 20X, which is a ton of wasted write operations and the reason why elaborate "garbage collection" algorithms have been developed to clean up data on SSDs after use so that they maintain performance.

 

[Decibel]

And yet, SandForce already provisions the most spare space of any of the controllers - 28% compared to the 6-7% Intel sets aside.

Then there is this potentially HUGE amount of additional unused space. Take the Win7 + Office = 11gb written for 25gb value SandForce themselves tout, fill the drive with similarly compressible data, so 44gb of the flash is used. This leaves 84gb of spare space - the 28gb set aside for performance and wear leveling and another 56gb lost in the magic of their "write less' algorithms.

Essentially, if you buy their marketing, (from the reviews there is no reason to doubt their claims) you're going to end up with a drive that significantly more than half of your expensive flash is unaccessible.

Like Bill said, this seems wasteful.

 

[Dezeer]

Essentially, if you buy their marketing, (from the reviews there is no reason to doubt their claims) you're going to end up with a drive that significantly more than half of your expensive flash is unaccessible.

Like Bill said, this seems wasteful.

Well that is what happens with write aplification and win 7 + office, you have 25GBs of data and the drive has to write only 11GBs.

But think that they would sell you a drive as 100GB and it would only be 64GB (50GiB as there is space taken by the controller), and you would start to save movies and pictures on to the drive.

Lets say Sandforce would get 0.95x aplification to those files. What happens when you have saved little less than 50GiB and you would like to save 50GiB more? There is no space to save any more files and you would start to lose the data.

I can think only one way that user could have "all" the space. A driver between the system and the controller so the OS knows how much the files take space, but that would f%&k up the idea behind the system and you would start to lose speed.

Correct me if I'm wrong.

 

[FrostBite]

You are correct. On the flip side, if write amplifitcaion was 10x or 20x, does that mean that your 100GB is now only 10GB or 5GB due to write amplifitcaion? Absolutely not. Write amplifitcaion is not space, write amplifitcaion is how often the flash memory is written on.

 

[Decibel]

Thanks guys, I get it. I guess I was thinking that it could be more flexible than it is, rather it is really conservative on how often it touches the flash.