Data at rest and unexpected power loss

[zz2]

Is data at rest at risk when ungraceful shutdown happens due to power loss with current consumer SSDs?

I remember it was an issue in the early days of SSDs, when unexpected power loss could corrupt data at rest. Then some companies wrote firmwares that could handle this, but since there are now many more manufacturers I'm interested, if now all modern SSD handle this? If not, which manufacturers do and which don't?

Note, that I'm talking only about power loss corruption that happens to data at rest, and not about data in flight which requires enterprise level of power loss protection.

 

[Maxx]

There's multiple ways to do this and actually I have sourced some patents on the technology, most specifically Crucial/Micron uses it in for example the MX500. In their case they use a differential device that can basically tell what the original bits (LSB, CSB) are, other methods rely on backing up these bits (internal buffers, DRAM, and sometimes in flash) with things that can evacuate in time if not using a battery or capacitor. I unfortunately can't locate the specific patents although they are part of my overall SSD archive, however I can tell you they work by reinstating/fixing the pages when the drive has its next power-on state. This is reinforced by the fact that consumer SSDs today rely on SLC caching which is of course much more robust, and when it later writes to native flash it doesn't delete/erase the original data until it gets confirmation, so therefore generally together these methods are sufficient to protect data at rest.

 

[Maxx]

Ah, here is one such patent. FYI, performance loss from such methods tend to be quite low, but it depends on the native flash (as this patent discusses).

I've spoken with Crucial engineers about this since they talked it up a lot with the P5 recently and they wouldn't give me a direct answer, however you can read the patents yourself and see exactly how they achieve this.

 

[zz2]

Thanks Maxx. I've got one MX500 because of it's power loss tech. I've seen Crucial/Micron advertise power loss protection for data at rest, but many other manufacturers provide little to none information regarding this issue for their SSD products.

For 4TB and larger SSDs cheaper brands look attractive, but I'm not sure about their reliability. Do you think it's safe to buy any brand?

 

[Maxx]

For consumer devices you will almost never have protection for data-in-flight, however for data-at-rest there are a number of methods used for mitigation (for example, SLC caching, as I mentioned above). Typically the lower pages (e.g. LSB when writing CSB) will be buffered internally (latches) or at least in memory (SRAM or DRAM) both of which are far faster than native flash so can evacuate more quickly to SLC. However if and when data corruption occurs there are other techniques based again on how flash is written as you can read here: since a superblock and superpage is open at once for writing, you can tell where the corruption is likely to occur and you can assume "worst-case" corruption when attempting a repair. This is of course based on the voltage because again you program page-by-page, well not exactly...you have interleaved page programming, but nevertheless the possible voltage states are known so you can estimate and re-program as necessary. Also as you can read from that source, it discusses the drawbacks of some methods (e.g. more writes) but the overall idea is to reduce the chance of errors/failure of recovery.

For example with MLC you can see that if we assume the lower page (LSB, first page written) was written correctly - which is true since each program operation will have a voltage verify pass - then if a value is outside that range (ability to be read) and we know from e.g. metadata that no upper page write was completed (due to power loss) than it's possible to rewrite the original value with some accuracy, done on next power-on. The difficulty here is that if the program was done in the distant past you can have retention effects (plus not every block is the same anyway), however with modern drives you tend to write sequentially and stale data is rewritten, retention impact from program disturb is minimal (due to 3D process node size being larger than 2D/planar), etc. Plus certain metadata will be tied to the block as in last written for example. So there's a ton of little aspects/nuances like this that an intelligent controller can use to be fairly resilient with data at rest.

In any case, to move away from the technical explanation: yes, all manufacturers have to some degree methods for protecting data-at-rest, whether it be a side-effect from the nature of writing or actual firmware techniques. Micron's statement for their drives as with the MX500, P2/P5, etc., specifically relate to the distinct differential memory device based on my communication with Crucial engineers. That is to say, they tell me it's specifically built into the flash/dies which makes it more robust than alternatives although in many cases this might not amount a direct improvement in failure rates so much as reducing the trade-offs therein. Which is to say, you can incur more writes which can reduce performance and endurance with more conservative methods of protecting data-at-rest, whilst their more expensive implementation can do it without that. An old patent I came across using actual data - that is, some space and writes to contain differential and/or journaling data that is compressed - could have any drive having ~5% performance impact with good resiliency (can't find it right now).

 

[zz2]

That's much more technical answer than I expected. Shows how complex is this technology really. So, all manufacturers have some methods of protecting data-at-rest, but the trade-offs are different depending on method used. Thank you Maxx for this comprehensive answer.

 

[Maxx]

Yep, data at rest is generally protected, although there are performance and endurance ramifications for it depending on how it is achieved. A couple years ago I couldn't say that with confidence but you mostly had 2D/planar MLC which was far less prone to issues from the get go. Now that we have 3D TLC and SLC caching, it's less of an issue than you would initially suppose. Older techniques would basically be compression of the differential (difference in voltage based on expectations) stored in flash which can take up a bit of space - not as much as you'd suspect because a superpage might only be 1MB for example - and it's actually fast since they can precharge the latches, e.g. if you have a data latch (DL, usually with the sense amplifier so "SADL") and cache latch ("XL") you can backup the differential quite easily. The since-expired patent I read said ~5% performance loss here and you will have more writes (write amplification) incl. on next power-on (when it refreshes), Micron's methodology seems to be better-integrated but along the same lines in terms of theory. However it's simple enough that any modern controller can manage protection of data at rest, data in flight is a different story but also less likely due to SLC caching.

I'm not an expert on the subject but the two patents linked above go into decent detail on how this works.