OpenSolaris derived ZFS NAS/ SAN (OmniOS, OpenIndiana, Solaris and napp-it)

Discussion in 'SSDs & Data Storage' started by _Gea, Dec 30, 2010.

  1. jad0083

    jad0083 Limp Gawd

    Messages:
    130
    Joined:
    Apr 30, 2006
    Yes. you should be able to flash it with an LSI 9211-8i IT firmware, to fully present the disks to the system (which ZFS prefers). With that said, that hba should be able to give you an additional 8 SATA/SAS hdds via 2x breakout cables, or even more via expanders.
     
  2. Nemesis_001

    Nemesis_001 n00bie

    Messages:
    43
    Joined:
    Apr 3, 2011
    Apologies, I mean 2 file systems on one pool which resides on one vdev.

    Meanwhile, the damn 8TB drives are stuck in customs.
     
  3. jad0083

    jad0083 Limp Gawd

    Messages:
    130
    Joined:
    Apr 30, 2006
    hey Gea,

    When you did your testing on optane as an slog, were you able to do any comparative testing on the different scsi drivers on esxi (lsi sas, pvscsi, nvme)?
     
  4. _Gea

    _Gea 2[H]4U

    Messages:
    3,736
    Joined:
    Dec 5, 2010
    I have used the default virtual disk driver (lsi)
     
  5. jad0083

    jad0083 Limp Gawd

    Messages:
    130
    Joined:
    Apr 30, 2006
  6. _Gea

    _Gea 2[H]4U

    Messages:
    3,736
    Joined:
    Dec 5, 2010
    new in napp-it 18.06 Dev (Apr 11)

    to be prepared for next OmniOS 151026 stable in may (or current bloody)
    https://github.com/omniosorg/omnios-build/blob/r151026/doc/ReleaseNotes.md

    Support for vdev removal (new ZFS feature) in menu Pools > Shrink Pool (OmniOS 151025+)
    Support for poolwide checkpoints (new ZFS feature) in menu Snaps > Checkpoint (OmniOS 151025+) and Pool > Import

    Disk Detection: adds ATTO HBAs (ATTO, a media specialist now supports Illumos)
    Disk Map: correct detection of HBA even if disks on different LSI mptsas HBAs shows the same c-number ex c0t500..
    Disk Map: add dd disk detection
    Disk Location: dd detection for all disks
     
    Last edited: Apr 11, 2018
  7. _Gea

    _Gea 2[H]4U

    Messages:
    3,736
    Joined:
    Dec 5, 2010
    The new OmniOS is the first Open-ZFS storage distribution to include the vdev remove (Pool shrink).
    Oracle Solaris 11.4 also comes with this feature but it seems with less restrictions.

    Open-ZFS, at least currently lacks the support for a vdev remove of a basic or mirror vdev when a raid-z [1-3] vdev is part of the pool or a remove raid-Z[1-3] at all or add a raid-z [1-3] after a remove of ex a basic/mirror vdev what limits its use cases. Support of raid-Z [2-3] is expected in Open-ZFS (but not Z1), Bug #7614: zfs device evacuation/removal - illumos gate - illumos.org

    Open-ZFS ex OmniOS that is the first to include this feature also requires a re-mapping table with a continous small RAM need/reservation and small performance degration.This is listed in the output of a zpool status. A manual zpool remap can fix this.

    It seems that Solaris 11.4 does not have these restrictions
    vdev removal, poolwide checkpoints/snaps or imp... | Oracle Community
     
    Last edited: Apr 12, 2018
  8. _Gea

    _Gea 2[H]4U

    Messages:
    3,736
    Joined:
    Dec 5, 2010
    Oracle Solaris 11.4 Beta Refresh is now available for download:
    https://community.oracle.com/message/14784683#14784683

    This refresh includes new capabilities and additional bug fixes.
    Some new features in this release:

    • ZFS vdev removal (all vdev types incl Raid-Zn)
    • ZFS Scheduled Scrub
    • SMB 3.1.1
     
    Last edited: Apr 28, 2018
  9. _Gea

    _Gea 2[H]4U

    Messages:
    3,736
    Joined:
    Dec 5, 2010
    A new snapshot of Openindiana Hipster 2018.04 is available

    OpenIndiana Hipster is a rolling distribution of the opensource Solaris fork Illumos with a snapshot every 6 months. It comes in the three flavours GUI with a Mate desktop, Text (very similar to OmniOS, another Illumos distribution) and Minimal.

    https://wiki.openindiana.org/oi/2018.04+Release+notes
     
  10. Pocatello

    Pocatello DC Moderator and [H]ard DCOTM x3 Staff Member

    Messages:
    5,996
    Joined:
    Jun 15, 2005
    I would like to create one of these but I don't know where to start. The user guide in the first post appears to be outdated.

    What software and OS is recommended?

    I would love to create a system using my current hardware which is older and outdated, but it can be given this new task. Once I get comfortable with the new OS and software I would like to build a second system with super-fast as my goal.

    Thanks for this thread!
     
  11. _Gea

    _Gea 2[H]4U

    Messages:
    3,736
    Joined:
    Dec 5, 2010
    OpenIndiana follows the idea of the former OpenSolaris as you can install a pure server version and a desktop capable version with GUI, many desktop apps and with a very wide use case range. OpenIndiana is a pure community project without a commercial support option.

    The main alternative is OmniOS with a single use case, ZFS storage server. It includes all what you need for that with a beta, stable and long term stable together with a commercial support option. There is no GUI option and the repository is strictly reduced to storage with a strong focus minimalism for a production quality stability.

    Setup, see 3.) at http://napp-it.org/manuals/index_en.html
     
  12. _Gea

    _Gea 2[H]4U

    Messages:
    3,736
    Joined:
    Dec 5, 2010
    OmniOS 151026 (may.07.2018) is out

    Release note: https://github.com/omniosorg/omnios-build/blob/r151026/doc/ReleaseNotes.md
    Download: https://downloads.omniosce.org/media/r151026/

    Main improvements:

    - Protection against the Meltdown Intel CPU vulnerability announced earlier this year
    - Experimental support for bhyve - a fast, lightweight and modern hypervisor
    - Sparse-branded zones, clocking in under 4MB per zone
    - An improved Installer which is dramatically faster making the OmniOS installation procedure
    one of the fastest in the industry. The new installer also provides many more options for
    customising the installed system.
    - A new lightweight default MTA (Dragonfly Mail Agent)
    - Fault management improvements for SSD disks

    ZFS features
    - Improved support for ZFS pool recovery (import good data from a damaged pool)
    - The new zfs remove of toplevel vdevs
    - zfs checkpoint features (poolwide checkpoint to make even a fs destroy/ vdev add/remove undoable)
    - Support for raidz2 and raidz3 boot disks

    Hardware
    - support of the new BroadCom tri-mode HBAs

    napp-it supports OmniOS 151026 up from 18.01 (apr.02)
    http://napp-it.org/downloads/changelog_en.html

    Update, see 3.)
    http://napp-it.org/manuals/index_en.html
     
    Last edited: May 8, 2018
  13. _Gea

    _Gea 2[H]4U

    Messages:
    3,736
    Joined:
    Dec 5, 2010
    ToddW2 likes this.
  14. CopyRunStart

    CopyRunStart Limp Gawd

    Messages:
    150
    Joined:
    Apr 3, 2014
    Hey Gea is it normal for the disks to show up twice when using Multi-Path? 2 HBAs, each with one cable connected to the JBOD.

    How do I add these disks to a pool? Which do I select of the two?

    EDIT: Fixed by adding my drive vendor and model to /etc/driver/drv/scsi_vhci.conf.
     
    Last edited: May 16, 2018
  15. gcooper

    gcooper n00bie

    Messages:
    6
    Joined:
    Feb 22, 2014
    I hope it is appropriate to post this here. My apologies if not.

    On OmniOS v11 r151016 and Napp-IT 0.9f6, I have three legitimately and physically removed drives displaying as "Removed". All pools are ONLINE (with new drives).

    How can I eliminate the removed drives from the display? A reboot? Trying to avoid that...

    Thanks in advance,

    G
     
  16. _Gea

    _Gea 2[H]4U

    Messages:
    3,736
    Joined:
    Dec 5, 2010
    The disks overview is based on iostat.

    Iostat keeps an inventory of disks and errors since bootup so remembering removed disks is as expected and as intended.
    This behaviour is essential if a disks suddenly fails and you need to know where it is or to check errors since last bootup.

    There are (unsupported) hacks to delete entries but in general only a bootup deletes the iostat memory.
     
    gcooper likes this.
  17. gcooper

    gcooper n00bie

    Messages:
    6
    Joined:
    Feb 22, 2014
    Perfect explanation. Thank you. G