What do you do to preserve your SSD performance on linux?

Discussion in 'SSDs & Data Storage' started by vFX, Dec 15, 2014.

  1. vFX

    vFX n00b

    Messages:
    55
    Joined:
    Sep 28, 2013
    nothing?
    noatime,discard in fstab?
    manual/scheduled fstrim command?

    something else?
     
  2. drescherjm

    drescherjm [H]ardForum Junkie

    Messages:
    14,627
    Joined:
    Nov 19, 2008
  3. KoolDrew

    KoolDrew [H]ard|Gawd

    Messages:
    1,646
    Joined:
    Oct 17, 2004
    Nothing. Should I be?
     
  4. devman

    devman 2[H]4U

    Messages:
    2,399
    Joined:
    Dec 3, 2005
    'discard' mount option on filesystems that support it should be sufficient.

    You should use 'noatime' or 'relatime' regardless of the media or filesystem. 'atime' was a bad idea when it was implemented, SSD didn't change that.
     
  5. drescherjm

    drescherjm [H]ardForum Junkie

    Messages:
    14,627
    Joined:
    Nov 19, 2008
    I thought about changing my nothing comment to discuss this. Regardless of the device I disable atime and diratime if possible.
     
  6. jwcalla

    jwcalla 2[H]4U

    Messages:
    3,629
    Joined:
    Jan 19, 2011
    discard for me

    and noatime
     
  7. cantalup

    cantalup Gawd

    Messages:
    758
    Joined:
    Feb 8, 2012
    additional on mine:
    make sure fstrim in crond that triggesr once a week for trimming.
     
  8. fantabulous

    fantabulous Limp Gawd

    Messages:
    229
    Joined:
    Jan 2, 2013
    discard + relatime or noatime, depending on the usage. I also enable discard in lvm.conf.
     
  9. danswartz

    danswartz 2[H]4U

    Messages:
    3,641
    Joined:
    Feb 25, 2011
    On a related note, I have a seagate SAS SSD I got for a ZFS SLOG device. It seems to be performing okay, but when I tried to trim it, I got this:

    Discarding device blocks: failed - Input/output error

    Looking in the log, I see a bunch of scsi reset messages. The SSD seems to be performing okay, so I'm not sure if there's an issue here. Is it okay to do a low-level scsi format to make sure everything is cleaned up?
     
  10. FnordMan

    FnordMan [H]ard|Gawd

    Messages:
    1,727
    Joined:
    Apr 22, 2011
    not necessary if the filesystem supports discard. (aka: trim)
     
  11. cantalup

    cantalup Gawd

    Messages:
    758
    Joined:
    Feb 8, 2012
    if you put discrard during mounting:
    whenever you delete files, kernel will call "trim" in real time. this coudl slow down while processing many files and daemons whom using files for writing and reading.

    fstrim is a lay back solution, that should be trigger on crontab(d) that do on one time, instead of files got deleting...

    which one do you prefer? pick one please...
    I prefer fstrim on crontab.
    two my systems running centos (server) and ubuntu(laptop) with ssd is using fstrim :D.
     
  12. cantalup

    cantalup Gawd

    Messages:
    758
    Joined:
    Feb 8, 2012
    is SSD behind expander?
     
  13. danswartz

    danswartz 2[H]4U

    Messages:
    3,641
    Joined:
    Feb 25, 2011
    Yes, but more research indicates it might be a known issue with some seagate SAS drives reporting one sector too many or something. I'm inclined to not mess with it unless given a reason to :)
     
  14. cantalup

    cantalup Gawd

    Messages:
    758
    Joined:
    Feb 8, 2012
    I believe that related with firmware issue..

    as I undestand on linux ,kernel just do what has bene told, unless range is specified
    oops... out of range....
     
  15. danswartz

    danswartz 2[H]4U

    Messages:
    3,641
    Joined:
    Feb 25, 2011
    I believe that, yeah. Unfortunately seagate does not yet have a firmware update for that SSD. I'm only using the first 16GB or so, so I am okay I think...
     
  16. SirMaster

    SirMaster 2[H]4U

    Messages:
    2,122
    Joined:
    Nov 8, 2010
    I use discard because fstrim always just says "0 bytes trimmed" so it appears to not be working.
     
  17. TGK

    TGK [H]Lite

    Messages:
    71
    Joined:
    Dec 5, 2005
    Crontabbed nightly fstrim on ext4 filesystem's.
     
  18. cantalup

    cantalup Gawd

    Messages:
    758
    Joined:
    Feb 8, 2012
    discard is handled automatically by kernel when enabled during mounting.
    fstrim is a "tool" to notify kernel to do trim manually by assuming discard is not in mount option.

    either one must be used, NOT both :D

    when t some files are deleted or truncated (as I my understandong on linux kernel), trim command executed by kernel if discrad is ENABLED or fstrim is EXECUTED.

    if you see "0 bytes trimmed", that is possible discrad is in mount option or No file deletion and truncation occures..
     
  19. SirMaster

    SirMaster 2[H]4U

    Messages:
    2,122
    Joined:
    Nov 8, 2010

    Sorry If I was not clear in my comment. Yes, I realize this.

    I do not use both. I used to use none. Then I learned about fstrim, but I could not get it to work. It was always "0 bytes trimmed" and it returns with this message instantly when I run it.

    Yes even after I delete a lot of files it's still 0 bytes. The disk I am trying to run fstrim on (Crucial M4 256GB) goes through about 5GB of create/delete per day so it should definitely be trimming something.

    So then since I can't get fstrim to work I mounted my disk with the discard option instead hoping that works.

    I have searched a long time and visited many IRC channels but nobody could tell me why fstrim is not working.

    My only guess to this point is that I am using an older version. "man fstrim" says November 2010 at the bottom of the man page. fstrim comes in util-linux package and when I check the version of util-linux it is 2.20. 2.25 is the latest, but I am using Debian Stable (Wheezy) and this is the newest package version and there is no newer version in wheezy backports. Only Debian Jessie (testing) but I cannot install that because of all sorts of package dependencies.
     
    Last edited: Dec 17, 2014
  20. JoeComp

    JoeComp [H]ard|Gawd

    Messages:
    1,036
    Joined:
    Jan 23, 2012
    What exact command and args are you calling fstrim with?

    What is the filesystem on the device you are trying to trim? Is the filesystem installed directly on the block device, or are there partitions or LVM?
     
  21. SirMaster

    SirMaster 2[H]4U

    Messages:
    2,122
    Joined:
    Nov 8, 2010
    1 SSD, 2 partitions, BTRFS and EXT4, no LVM.

    EXT4 is working with fstrim, but BTRFS is not. I have read all around the web that fstrim works on BTRFS however.

    Code:
    root@nick-server:~# fstrim / -v
    /: 0 bytes were trimmed
    
    root@nick-server:~# fstrim /mnt/misc -v
    /mnt/misc: 405372834 bytes were trimmed
    
    Code:
    root@nick-server:~# mount
    ...
    /dev/sda1 on / type btrfs (rw,relatime,ssd,space_cache)
    /dev/sda2 on /mnt/misc type ext4 (rw,noatime,errors=remount-ro,data=ordered)
    ...
    
    Code:
    root@nick-server:~# fdisk -l /dev/sda
    
    Disk /dev/sda: 256.1 GB, 256060514304 bytes
    255 heads, 63 sectors/track, 31130 cylinders, total 500118192 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x0006fbdb
    
       Device Boot      Start         End      Blocks   Id  System
    /dev/sda1            2048   209717247   104857600   83  Linux
    /dev/sda2       209717248   500117503   145200128   83  Linux
    
    
    Code:
    root@nick-server:~# parted /dev/sda print
    Model: ATA M4-CT256M4SSD2 (scsi)
    Disk /dev/sda: 256GB
    Sector size (logical/physical): 512B/512B
    Partition Table: msdos
    
    Number  Start   End    Size   Type     File system  Flags
     1      1049kB  107GB  107GB  primary  btrfs
     2      107GB   256GB  149GB  primary  ext4
    
    
     
    Last edited: Dec 17, 2014
  22. JoeComp

    JoeComp [H]ard|Gawd

    Messages:
    1,036
    Joined:
    Jan 23, 2012
    Interesting that it works on ext4 but not btrfs for you.

    I have used fstrim on XFS and ext4 filesystems, and it works as expected. I have not tried btrfs.

    If your version of fstrim is old, that might explain why it does not work on btrfs. btrfs is such a moving target.
     
  23. SirMaster

    SirMaster 2[H]4U

    Messages:
    2,122
    Joined:
    Nov 8, 2010
    Yeah, I wish I knew a way to update util-linux without breaking my whole machine.

    I'm OK with the discard mount option solution, but I have no way of knowing if it's actually working :\
     
  24. JoeComp

    JoeComp [H]ard|Gawd

    Messages:
    1,036
    Joined:
    Jan 23, 2012
    I'm not even sure how exactly TRIM is supposed to work on a COW filesystem like btrfs in cases where you have snapshots. If you delete a file in a directory that is part of a snapshot, I suppose fstrim would not free any sectors. And there may be other complications I have not thought of.
     
    Last edited: Dec 17, 2014
  25. reb00tin

    reb00tin Gawd

    Messages:
    864
    Joined:
    Apr 13, 2009
    noatime,discard in fstab

    I don't bother with scheduler, kernel takes care of that, since you could have a mix of SSD & rotating disks.
     
  26. cantalup

    cantalup Gawd

    Messages:
    758
    Joined:
    Feb 8, 2012
    as I understand,
    if fstrim does not work, discard is not working too, since fstrim triggers kernel to do send trim execution(command).
     
  27. cantalup

    cantalup Gawd

    Messages:
    758
    Joined:
    Feb 8, 2012
    taken from your link ....
    https://btrfs.wiki.kernel.org/index.php/FAQ
    Does Btrfs support TRIM/discard?
    ......
    There are two ways how to apply the discard:

    * during normal operation on any space that's going to be freed, enabled by mount option discard
    * on demand via the command fstrim
    ....

    I suspect your kernel does not support trim on btrfs.
    I believe kernel 3.X.X that has stable btrfs raid0/1 alike, supports trim on brfs
    .

    exactly, waiting btrfs to support raid5/6 (raidz1/raidz2) to replace my current ZoL,
    I prefer more intergrated solution than DKMS in ZoL :D
    this is pain in the butt, when updating kernel, since need to rebuild DKMS (always failed automaticaly rebuild on my experience as today).

    stick on xfs or ext4 D...

    I have centos 7 (got to learn new systemd....love and hate) that fstrim work on btrfs.
     
    Last edited: Dec 17, 2014
  28. cantalup

    cantalup Gawd

    Messages:
    758
    Joined:
    Feb 8, 2012
    two solution, and please pick one :p...

    kernel is smart enough to know SSD :D...
    if you run fstrim on non SSD, nothing happens as I know.

    SSD is already in linux kernel mainstream.
     
  29. cantalup

    cantalup Gawd

    Messages:
    758
    Joined:
    Feb 8, 2012
    as I understand, trim is lower level than filesystem such as: xfs, btrfs, ext4, etc.
    if the filesystem deletes some space (not related with deleting same file in snapshot which is not real deletion in low level), trim will reclaim as free memory/space.

    SSD does not have sectors :D.
    trim just reclaim free (deleted) memory.

    please correct me.........
     
  30. SirMaster

    SirMaster 2[H]4U

    Messages:
    2,122
    Joined:
    Nov 8, 2010
    So if TRIM isn't working either way what options do I have? heh.

    What exactly are the consequences I should expect of not having a functional TRIM over time. I've had this system running for 6 months so far. Something about slower performance, right? But is really that bad? Does it eventually hit a bottom point or does it get worse to infinity?
     
  31. JoeComp

    JoeComp [H]ard|Gawd

    Messages:
    1,036
    Joined:
    Jan 23, 2012
    You are confusing internal SSD structure with the OS and filesystem interface to the device. A filesystem does NOT "delete some space". The filesystem is responsible for keeping track of where on the block device data is stored, but outside of TRIM, a filesystem does not bother to "delete". It just overwrites blocks that are no longer allocated to files.

    An SSD does have logical sectors (aka LBAs), like all ATA devices. That is the interface (API, if you will) that the SSD presents to the OS.

    I haven't looked at the fstrim code, but it is obvious that fstrim must somehow inform the filesystem either that the filesystem needs to TRIM the unused sectors, or else the filesystem must inform fstrim what sectors may be TRIMmed and then send the TRIM commands itself. Either way, you have two key things that must take place:

    (1) the filesystem must identify the sectors (LBAs) that may be TRIMmed

    (2) the TRIM command must be sent to the SSD for those sectors

    Since fstrim, when it fails for SirMaster, appears to know that nothing has been TRIMmed, my guess is that the failure is occurring in step (1). Although I suppose it is still possible that step (2) occurs and fstrim recognizes the error message and so knows that nothing has been TRIMmed, I guess that is not the problem.
     
    Last edited: Dec 17, 2014
  32. JoeComp

    JoeComp [H]ard|Gawd

    Messages:
    1,036
    Joined:
    Jan 23, 2012
    I'd say your options are:

    (1) Switch to ext4 or XFS

    (2) Update to a newer kernel and to the latest fstrim

    (3) Secure erase your SSD, then repartition except leave 10-25% of the space unpartitioned (i.e., overprovision the SSD)

    How much of a slowdown you get depends sensitively on the workload, and of course on the SSD itself. As does how effective OP will be and how much OP you need.
     
  33. cantalup

    cantalup Gawd

    Messages:
    758
    Joined:
    Feb 8, 2012
    thanks for correction...
    "filesystem delete some space"-> mark as free.this depends on the filesystem structure, as I know using tree algorithm...
    logical sectors. just got too much kool-aid(low level linux works that I mix-up) at my works
    you can verbosely set the range on fstrim.

    basically, I correct my wording, in ssd level, there is no sectors, just memory location, on ssd interface (SATA), logical level is introduced to outside world.

    basically, fstrim just rely on kernel mechanisem to do trim.
    fstrim is lay back interface to us since we are in control to do trimming.

    source codes can be read on linux kernel starting with 3.X.X (do not remember minor version).
     
    Last edited: Dec 17, 2014
  34. cantalup

    cantalup Gawd

    Messages:
    758
    Joined:
    Feb 8, 2012

    not really bad.... still works
    my suggestion:
    1)plan a get away solution, when you are seeing slowness in your btrfs partition(SSD),
    ex: convert to xfs or ext4, and copy back the data
    2) update kernel that supports btrfs with trim. this is taking down-time longer since need to get the correct kernel, and do upgrade. this usually would drag some other problems too.

    if the machine is very important, I would take "1" solution.
    do you want a blue or read pill?..
     
  35. SirMaster

    SirMaster 2[H]4U

    Messages:
    2,122
    Joined:
    Nov 8, 2010
    Well the partition is 100GB and I only use about 10GB of it and this isn't really going to change.

    I really don't want to switch filesystems because I use BTRFS in order to take snapshots so I can do full and consistent backups of my root.

    Upgrading of the kernel is not really possible. I am running debian wheezy and some software I run depends on that. I am already using the backported kernel which is currently 3.16.7 and it gets updated every couple months, so eventually that will be updated. 3.16 is pretty new, I'm not even sure a newer kernel would change anything though since btrfs has supported TRIM since kernel 2.6 days.

    I wish there was a way to confirm if it is working with discard.
     
  36. jwcalla

    jwcalla 2[H]4U

    Messages:
    3,629
    Joined:
    Jan 19, 2011
  37. JoeComp

    JoeComp [H]ard|Gawd

    Messages:
    1,036
    Joined:
    Jan 23, 2012
  38. jwcalla

    jwcalla 2[H]4U

    Messages:
    3,629
    Joined:
    Jan 19, 2011
    No, but the link does mention that if it is read as zeros it's working.
     
  39. cantalup

    cantalup Gawd

    Messages:
    758
    Joined:
    Feb 8, 2012
    you look the source code on linux kernel. start with 3.XX version..

    non zeroes basically, kernel has no response/null return, when trim is done or timeout.

    it should NON-zeroes return by assuming there is some activities and discard is no in mount opttion (fstab).

    I believe, not all ssd created equally, that could be some ssd return null or timeout during trim command from kernel.

    picked well know ssd to play safe:D.
    or mostly SSD firmware is a culpirt, since ssd supports is already mature in main-strem linux kernel.
     
  40. SirMaster

    SirMaster 2[H]4U

    Messages:
    2,122
    Joined:
    Nov 8, 2010
    The point is moot anyways.

    The commands don't work on BTRFS.

    Code:
    root@nick-server:/# hdparm --fibmap tempfile
    0,17: device not found in /dev
    
    I did do the process on my EXT4 partition and the sectors do end up becoming 00 after the file is deleted and synced.

    But on BTRFS I have no way (that I know of) to locate the file sectors.

    --read-sector works, but --fibmap does not.
     
    Last edited: Dec 18, 2014