Is my local datastore SSD dying?

D

danswartz

2[H]4U
Joined
Feb 25, 2011
Messages
3,715
My ESXi 5.0 box has a 64GB crucial M4 as local datastore (system runs from USB stick.) Local datastore has astaro security gateway (low/moderate disk traffic) and openindiana SAN (low disk traffic) on it. The ASG doesn't *really* need to be there; I had it moved there when the SAN was a separate box (now an all in one) and I was having to fiddle with the external SAN and didn't wan the internet 'going down the time'. Anyway, lately, I've noticed a lot of messages like this:

2012-08-28T10:41:30.760Z cpu5:2053)WARNING: ScsiDeviceIO: 1218: Device t10.ATA_____M42DCT064M4SSD2__________________________0000000011500324D633 performance has deteriorated. I/O latency increased from average value of 5793 microseconds to 597734
2012-08-28T10:41:44.974Z cpu6:2054)WARNING: ScsiDeviceIO: 1218: Device t10.ATA_____M42DCT064M4SSD2__________________________0000000011500324D633 performance has deteriorated. I/O latency increased from average value of 5963 microseconds to 120223
2012-08-28T10:41:47.295Z cpu6:2054)ScsiDeviceIO: 1198: Device t10.ATA_____M42DCT064M4SSD2__________________________0000000011500324D633 performance has improved. I/O latency reduced from 1202236 microseconds to 233754 microseconds.
2012-08-28T10:41:50.071Z cpu2:2050)ScsiDeviceIO: 1198: Device t10.ATA_____M42DCT064M4SSD2__________________________0000000011500324D633 performance has improved. I/O latency reduced from 233754 microseconds to 46077 microseconds.
2012-08-28T10:44:40.036Z cpu6:3420)ScsiDeviceIO: 1198: Device t10.ATA_____M42DCT064M4SSD2__________________________0000000011500324D633 performance has improved. I/O latency reduced from 46077 microseconds to 11662 microseconds.

This is an SSD connected to mobo SATA port (no other SATA clients on mobo). Is the drive flaking out, or do I have some software issue? Hard to imagine I/O latency over 1/2 second for an SSD? Almost forgot: I also have ESXi configured to log to that SSD..
 
Depending on TRIM support (which there isn't in your case) and how the drive manages garbage collection 1/2 second latencies are not at all out of the question for a SSD.

I would keep an eye on it, but depending on your use pattern a larger SSD partitioned with more spare area would likely help.

Viper GTS
 
Hmmm, thanks. Well, out of the 64GB. about 1/4 of it is unused. If I move the firewall to the SAN (now that that is stable), that will likely eliminate most/all of this. Not even sure how I would manually partition a disk to leave some spare for ESXi (I assume I'd have to hack on the CLI.) I think I'm going to start by moving the ASG appliance and see what effect that has. Thanks!
 
Local controller? On the mobo? It's an x9scl-f mobo, so a cougar point controller. I have not set any kind of custom logging rules, so whatever the esxi 5.0 default is...
 
Over the last 4 days (since vmkernel.log rolled), 34 instances. Not a lot, I would guess, more concerned this indicates an issue. I just upgraded the ram from 16 to 32, so I moved the ASG off to the SAN datastore. No recurrences yet, but it's only been a couple of hours.
 
Literally all it means is that the latency increased for a few commands. Given how MLC flash works, I wouldn't be worried - might just be forced garbage collect.

Remember, use doesn't determine garbage collect (other than load over time) - I can put 4M on a disk, and eventually make it go into garbage collect. Looks like it went for about 3 seconds.
 
You must log in or register to reply here.
Back
Top