asking for friend: which filesystem for 6 TB HDD?

Discussion in 'Linux/BSD/Free Systems' started by x509, Jul 8, 2018.

  1. x509

    x509 [H]ard|Gawd

    Messages:
    1,585
    Joined:
    Sep 20, 2009
    I just helped a friend build a new system based on a Gigabyte XZ270 motherboard, with a Samsung M2. SSD and a 6 TB HGST HDD. (I know almost nothing about Linux, although in a previous life I worked with UNIX and Solaris.)

    He installed Linux Mint 19 and then wanted to create a filesystem on the H6 TB HDD. He ran fdisk to format the 6 TB drive, but it would create only a 1.8 TB partition. He wants a single partition and filesystem for the entire HDD.

    If this was Windows, I would know exactly what to do, but it's Linux, so i know nothing. So what do I tell my friend to do?

    x509
     
  2. Mazzspeed

    Mazzspeed Gawd

    Messages:
    856
    Joined:
    Dec 27, 2017
    Usually just Ext4. What's the HDD to be used for?
     
  3. ChadD

    ChadD I Love TEXAS

    Messages:
    3,088
    Joined:
    Feb 8, 2016
    fdisk will never allow you to create a partition larger then 2TB, it uses a MS-DOS partition table and simply is a no go for formatting large single parts.

    You (they) will need to use parted or gdisk.
    http://dev-random.net/creating-mounting-partition-lager-2-tb-linux/

    As for file system.

    EXT4 is in general a safe all around good choice.
    https://en.wikipedia.org/wiki/Ext4

    XFS is another good option for large drives. (although most distros use EXT4 some such as SUSE use XFS as the default for data /home and data partitions)
    https://en.wikipedia.org/wiki/XFS

    Here is a comparison of Linuxs 3 most popular file systems. Ext4 XFS and Btrfs... I wouldn't use btrfs for a data drive as you won't like need most of its features such as copy on write which are more suited for OS files imo. (bring Suse up again their default is BTFS for /root and XFS for /home)
    http://www.electronicdesign.com/ind...-between-linux-ext-xfs-and-btrfs-file-systems

    Here is a fairly recent performance breakdown of the major file systems.
    https://www.phoronix.com/scan.php?page=article&item=linux414-fs-compare&num=1
    In most of these you will see in situations involving large files XFS tends to be the fastest... Ext4 is likely the best all around file system. And BTFS really solves a difference set of issues.

    Anyway EXT4 or XFS... doesn't much matter and use a capable too like parted to setup the partition.

    PS... as for the instructions on Gparted... should work well. However seeing as you mentioned your friend installing Mint. If their Linux fu is not so strong... and the gparted terminal stuff is no go. Mint I believe ships with Gnome Disks. Disks shouldn't have any issue formatting larger disks. Its been awhile but I believe it asks if you want to use, "Compatible with modern systems and hard disks (GPT)" or something like that. Won't have the same limitations as fdisk... and its GUI.
     
    Last edited: Jul 9, 2018
  4. B00nie

    B00nie [H]ardness Supreme

    Messages:
    7,425
    Joined:
    Nov 1, 2012
    Had to check if that URL actually had a spelling error - and indeed it does. Funny.
     
    ZeqOBpf6 likes this.
  5. ChadD

    ChadD I Love TEXAS

    Messages:
    3,088
    Joined:
    Feb 8, 2016
    lol ya I just linked something I googled quick. :)
     
  6. x509

    x509 [H]ard|Gawd

    Messages:
    1,585
    Joined:
    Sep 20, 2009
    General storage. He is a consultant and needs to stores various client files.

    x509
     
  7. Mazzspeed

    Mazzspeed Gawd

    Messages:
    856
    Joined:
    Dec 27, 2017
    My advice is just to use Ext4, as is it's a superior file system to NTFS.
     
  8. ChadD

    ChadD I Love TEXAS

    Messages:
    3,088
    Joined:
    Feb 8, 2016
    Ok this is perhaps a stupid question days later but I guess no one asked.

    What File system did he use for his system disk ? ext4 or Btrfs.

    Mint just updated their backup tool... I have no idea what FS they default to anymore.

    However having said that... tell him to take a look at Mint Backup util and Timeshift. They have added a bunch of features.

    Seeing as this is client data... it may be wise to use Btrfs actually. His mint utilities may do automatic snapshots, which he could use to easily back up client data.

    BTFS supports copy on write, I glossed over it earlier. But what it means in a nut shell is snapshots of the entire disk are created pretty much instantly. Making it very nice if you are backing things up often. (its also pretty easy to automate that... with BTFS setup its not some big long process to back things up.)

    Even if he isn't using the new Mint tools... which I admit knowing little about. Using btrfs for commercial storage drives is nice as you can use simple tools like Snapper to setup auto snapshots. Its not as complicated as it sounds.
     
    Last edited: Jul 13, 2018
  9. x509

    x509 [H]ard|Gawd

    Messages:
    1,585
    Joined:
    Sep 20, 2009
    I'm the OP. Not sure what the system disk was formatted as. It's a 500 GB Samsung M.2 SSD.
     
  10. ChadD

    ChadD I Love TEXAS

    Messages:
    3,088
    Joined:
    Feb 8, 2016
    Most likely they installed EXT4. Last time I used mint was some time ago but I don't remember it doing anything odd with its default file system suggestion. So I'm sure it still just uses EXT4. Which is a good speedy reliable FS.

    For your friends 6 TB drive. Follow the other stuff we posted to make it one big drive if that is what they want. fdisk won't work but the other Linux tools will work. Parted, gpart... and Gnome Disks can do it all from a GUI (pretty sure it comes pre installed on Mint, if not its easy to install just search disks in their package manager.

    As for backups... I'm not sure what they are planning to do.
    Btrfs has some advantages for backups, again I would suggest you just point them to the info or give it a good ready yourself.
    https://en.wikipedia.org/wiki/Btrfs

    Its possible btrfs makes sense if they are planning to back that data up fairly often. A well setup btrfs backup setup can do things like automatically backup to remote servers... or a different volume if they want to setup auto back up to a USB backup drive or something. You can do most of that with software and any file system... BTRFS just makes it a lot faster and seemless, a setup using incremental backups using btrfs is in general much faster then other options.

    If you have heard of ZFS... btrfs shares a lot of the features. ZFS is used by a lot of large data centers... and most NAS type products.
    https://en.wikipedia.org/wiki/ZFS
    I wouldn't use ZFS on Linux... its not mainlined in the kernel, you can make it work. But btrfs is the way to go if you want ZFS type features on linux.
     
  11. B00nie

    B00nie [H]ardness Supreme

    Messages:
    7,425
    Joined:
    Nov 1, 2012
    Last time I checked reviews and benchmarks, Ext was the only truly stable fs. BTFS and ZFS were far more problematic. Maybe this has changed.
     
  12. ChadD

    ChadD I Love TEXAS

    Messages:
    3,088
    Joined:
    Feb 8, 2016
    btrfs has been in the stable line for years now. When talking about a Personal PC btrfs is not just extremely stable it may be the most stable FS in the mainline kernel. Even when it was still not labeled "stable" that was true. It was labeled unstable due to issues with specific raid settings ect. As it is aimed at large data center machines it took awhile to get a "stable" sign off.

    ZFS as I say its not really a Linux file system. Its a BSD file system... and one of the most stable high end FS around. You find ZFS on almost all commerical NAS products.. and its the FS used by systems like Freenas.

    Still ya if you aren't going to use the features of btrfs... its better to stick to ext4.
     
  13. B00nie

    B00nie [H]ardness Supreme

    Messages:
    7,425
    Joined:
    Nov 1, 2012
    I wasn't talking about 'stable line' but stable as in reality. IIRC btfs had a tendency to croak irreversebly in certain situations, requiring a total rebuild.
     
    Mazzspeed likes this.
  14. Mazzspeed

    Mazzspeed Gawd

    Messages:
    856
    Joined:
    Dec 27, 2017
    This is why I didn't mention btrfs, especially considering the OP's obviously limited knowledge of Linux. No disrespect to the OP. :)
     
    B00nie and x509 like this.
  15. tbg

    tbg Limp Gawd

    Messages:
    140
    Joined:
    Jul 30, 2013
    ext4 will work fine on the 6tb disk. As ChadD said, you want to use parted so you can create a gpt partition table so you can use the whole disk.

    you mentioned client files, depending on how much down time he can take in the event of a disk failure, he may want to consider having 2 drives and mirrior them. the drives can be mirrored with mdadm and you can still create the ext4 filesystem on top of that. obviously they should still have a separate back up plan as raid is not a substitute for backups.

    zfs on linux is pretty decent, we have a sizable box at work running it. however being inexperienced with linux its probably best to stick with the common fs types like ext4, so you are more likely to be able to get help with a google search.
     
  16. x509

    x509 [H]ard|Gawd

    Messages:
    1,585
    Joined:
    Sep 20, 2009
    My friend does backups every day, plus an offsite backup.
     
  17. Mazzspeed

    Mazzspeed Gawd

    Messages:
    856
    Joined:
    Dec 27, 2017
    Back In Time works really well for backups.
     
  18. Frobozz

    Frobozz [H]ard|Gawd

    Messages:
    1,714
    Joined:
    Feb 15, 2002
    I think that's if you utilize any* of the RAID like qualities. OpenSUSE has been using BTRFS as their default filesystem for / for quite some time and it affords some ass saving features (snapshot rollback through snapper). I know "It works fine for me" is kindofa crap statement, but it's worked well for me for a few years with Tumbleweed.

    All that said, for a storage drive of important stuff, I'd probably still go with LVM + EXT4. EXT4's toolset is robust (you can expand and shrink volumes. Last time I tried, you couldn't shrink XFS which would be my next runner up.) Hard to go against something tried and true even if it feels a bit vanilla.

    * = I think a lot of RAID 5 type functionality. I'm not sure if this applies to mirror configs and such.
     
    ChadD likes this.
  19. B00nie

    B00nie [H]ardness Supreme

    Messages:
    7,425
    Joined:
    Nov 1, 2012
    The snapshot ability is quite enticing, but I'm just writing on my past information. It may be outdated.
     
  20. naib

    naib [H]ard|Gawd

    Messages:
    1,056
    Joined:
    Jul 26, 2013
    I would seriously question that stance... "Stable" as in mainline sure "stable" as in on-disk format sure. "stable" as data retention? no...
    Have a read of this thread: https://forums.gentoo.org/viewtopic-t-1081990.html

    Right now I would say Ext4, proven real-world and is continually being tweaked for performance and stability. I would however tunefs to reduce the root allocation. By default there is 1% reserved so if the drive fills up ONLY root can login to free up space. at 6Tb that is a very large reserve :) I have reduced this on my 1Tb M.2 drive .

    There are three "next Gen" files systems

    1) ZFS
    2) BTRFS
    3)Bcachefs

    #1 is license incompatible BUT openZFS keeps in lock-step. I keep away from this due to out of tree (see the thread I posted for my reasons)
    #2 risky... even today
    3) Now this looks REALLY interested but not quite there yet...

    tl;dr Ext4 :)