OpenSolaris derived ZFS NAS/ SAN (OmniOS, OpenIndiana, Solaris and napp-it)

Ever since I went to the latest stable omniOS I am getting weird file modified behavior. If I make a VBS file in notepad++ and keep it open while I execute the script, I get a prompt saying the file has been modified. It used to not modify the file after I executed it. Any idea what's going on here?
is atime disabled?
 
OmniOS v151038
Napp-it v19.12

Gea,

I'm updating my hardware to a Dell R720 with the 3.5 bays up front. Looking to get your opinion on drives and how they are setup for the SLOG, OmniOS, and the VM storage pieces please.

New server specs: Dell R720
Memory: 96GB RAM
CPU: (2) Intel Xeon E5-2600 series
Dell H710P flashed to IT Mode
(4) 8TB WDC SATA for main large pool

other parts lying around
(1) Samsung 983 DCT U.2 SSD (within my budget)
(1) WD-Black SN750 NVMe SSD
(1) Intel Optane M.2 80mm PCIe 3.0 32GB SSD
(also can use the SSDs in my current setup)

I'm currently running ESXi 6.5 as my boot OS and have a separate SSD drive presented to ESXI for OmniOS. Also I have a mirror with two SSDs for my VM area that is running out of space.

The zpool status below is my setup currently running on an aging Supermicro board. I plan on moving over the raidz1-0 pool and backup the OmniOS VM to move over to my new hardware.

Plan so far for new hardware setup:
1. Install ESXI 7.x on a USB Flash drive
2. Move over the current raidz1-0 pool
3. backup the OmniOS VM and reuse
4. Use the Samsung 983 DCT for the SLOG

Questions:
1) Should I use a separate SSD just for OmniOS?
2) Is it a good idea to create a pool just for where my VMs will live or should these run on faster drives than just the 7200RPM SATA disks that are in the raidz1 pool?
3) have I missed anything, or can you provide any suggestions?

Thank you so much for all of help you've given over the years - greatly appreciated!


Code:
zpool status  

  pool: NAPPTANK
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
    still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
    the pool may no longer be accessible by software that does not support
    the features. See zpool-features(5) for details.
  scan: scrub repaired 0 in 1 days 07:49:09 with 0 errors on Tue Jul  6 06:49:12 2021
config:

    NAME                       STATE     READ WRITE CKSUM      CAP            Product /napp-it   SN/LUN           IOstat mess       SMART
    NAPPTANK                   ONLINE       0     0     0
      raidz1-0                 ONLINE       0     0     0
        c3t5000CCA26BD16F32d0  ONLINE       0     0     0      8 TB           WDC WD80EMAZ-00W   S:0 H:0 T:0       -
        c3t5000CCA266F7BC19d0  ONLINE       0     0     0      8 TB           WDC WD80EMAZ-00W   S:0 H:0 T:0       -
        c3t5000CCA252C83AD8d0  ONLINE       0     0     0      8 TB           WDC WD80EMAZ-00W   S:0 H:0 T:0       -
        c3t5000CCA26BC80AECd0  ONLINE       0     0     0      8 TB           WDC WD80EMAZ-00W   S:0 H:0 T:0       -
    logs  
      c3t55CD2E404B6E030Fd0    ONLINE       0     0     0      100 GB         INTEL SSDSC2BA10   S:0 H:0 T:0       -

errors: No known data errors

  pool: SSD_MIRROR
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
    still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
    the pool may no longer be accessible by software that does not support
    the features. See zpool-features(5) for details.
  scan: scrub repaired 0 in 0 days 00:12:43 with 0 errors on Sun Jul  4 23:12:46 2021
config:

    NAME                       STATE     READ WRITE CKSUM      CAP            Product /napp-it   SN/LUN           IOstat mess       SMART
    SSD_MIRROR                 ONLINE       0     0     0
      mirror-0                 ONLINE       0     0     0
        c3t50025388A07E6676d0  ONLINE       0     0     0      512.1 GB       Samsung SSD 850    S:0 H:0 T:0       -
        c3t5002538840224D6Dd0  ONLINE       0     0     0      512.1 GB       Samsung SSD 850    S:0 H:0 T:0       -

errors: No known data errors

  pool: rpool
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
    still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
    the pool may no longer be accessible by software that does not support
    the features. See zpool-features(5) for details.
  scan: none requested
config:

    NAME        STATE     READ WRITE CKSUM      CAP            Product /napp-it   SN/LUN           IOstat mess       SMART
    rpool       ONLINE       0     0     0
      c2t0d0s0  ONLINE       0     0     0      32.2 GB        Virtual disk       S:0 H:0 T:0       -

errors: No known data errors

Code:
current disks:

 id                              part       identify       stat       diskcap       partcap       error       vendor       product       sn       temp       smart_overview2
 c1t0d0       -       -       removed               -        S:0 H:5 T:0       NECVMWar       VMware IDE CDR10               -       -
 c2t0d0       single       via dd        ok               32.2 GB        S:0 H:0 T:0       VMware       Virtual disk              -       -
 c3t5000CCA252C83AD8d0       single       via dd        ok               8 TB        S:0 H:0 T:0       ATA       WDC WD80EMAZ-00W              -       -
 c3t5000CCA266F7BC19d0       single       via dd        ok               8 TB        S:0 H:0 T:0       ATA       WDC WD80EMAZ-00W             -       -
 c3t5000CCA26BC80AECd0       single       via dd        ok               8 TB        S:0 H:3 T:2       ATA       WDC WD80EMAZ-00W             -       -
 c3t5000CCA26BD16F32d0       single       via dd        ok               8 TB        S:0 H:0 T:0       ATA       WDC WD80EMAZ-00W              -       -
 c3t5002538840224D6Dd0       single       via dd        ok               512.1 GB        S:0 H:0 T:0       ATA       Samsung SSD 850              -       -
 c3t50025388A07E6676d0       single       via dd        ok               512.1 GB        S:0 H:0 T:0       ATA       Samsung SSD 850            -       -
 c3t55CD2E404B6E030Fd0       single       via dd        ok               100 GB        S:0 H:0 T:0       ATA       INTEL SSDSC2BA10
 
Last edited:
OmniOS v151038
Napp-it v19.12

Gea,

I'm updating my hardware to a Dell R720 with the 3.5 bays up front. Looking to get your opinion on drives and how they are setup for the SLOG, OmniOS, and the VM storage pieces please.

New server specs: Dell R720
Memory: 96GB RAM
CPU: (2) Intel Xeon E5-2600 series
Dell H710P flashed to IT Mode
(4) 8TB WDC SATA for main large pool

other parts lying around
(1) Samsung 983 DCT U.2 SSD (within my budget)
(1) WD-Black SN750 NVMe SSD
(1) Intel Optane M.2 80mm PCIe 3.0 32GB SSD
(also can use the SSDs in my current setup)

I'm currently running ESXi 6.5 as my boot OS and have a separate SSD drive presented to ESXI for OmniOS. Also I have a mirror with two SSDs for my VM area that is running out of space.

The zpool status below is my setup currently running on an aging Supermicro board. I plan on moving over the raidz1-0 pool and backup the OmniOS VM to move over to my new hardware.

Plan so far for new hardware setup:
1. Install ESXI 7.x on a USB Flash drive
2. Move over the current raidz1-0 pool
3. backup the OmniOS VM and reuse
should work
4. Use the Samsung 983 DCT for the SLOG
should be slightly faster than the curent DC 3700
Questions:
1) Should I use a separate SSD just for OmniOS?
You need a bootdrive for ESXi (USB or Sata) and a local datastore for OmniOS. I usually use an Sata SSD (60GB+) for both - ex the DC 3700
2) Is it a good idea to create a pool just for where my VMs will live or should these run on faster drives than just the 7200RPM SATA disks that are in the raidz1 pool?
Common is a large disk based pool for Filer/Backup and a second faster SSD based pool for VMs.
3) have I missed anything, or can you provide any suggestions?

Thank you so much for all of help you've given over the years - greatly appreciated!
thanks, you are welcome
 
is atime disabled?
It wasn't but I disabled it and the issue persisted. Finally figured it out today. Turns out the napp-it vm was 4 mins ahead of time so everything was just constantly thinking it was modified but from the future. This napp-it vm is domain joined so I thought it would NTP sync automatically with the DC but I guess that isn't the case. I manually set the time to be correct and its working now.
 
Completely unrelated, but is there a way to configure file access audits in napp-it? I know solaris can do it but not sure if omnios can
 
I'm currently using ESXI 6.5 with Napp-it + Omni as VM, should I upgrade to ESXI 6.7?

Thanks
 
New default napp-it is v 21.06 for all releases incl. free
Free is ok for commercial use and without a capacity limit but without support or updates to dev/support releases or PRO addons
 
_Gea Could I get an opinion from you? How would you handle less important *ISOs* mixed with personal photos and such, and some CPU for converting said *ISOs* around while supporting a few game servers?
I have 8x8TB drives right now in a mergerfs and I'm irritated by it more than anything. My important stuff sits on a Raid Z2 with some aging and close to full drives but that's less of an issue. I'm planning to replace my aging 2650v2 setup with something more modern, like a 5800x with some ECC, but I'm not terribly picky. Trying to cut back my idle 300w usage to something more reasonable while adding some capability to the system. I'd rather not spend over 1k on this upgrade (not including HDDs), but it's flexible.

If you had to build your ideal server, with or without hot swap (and hot spares), what would you shoot for today? My goal is to keep the 8x8TB in use, splitting 5 into a Raid Z2 with a hot spare on top for important data (to replace the aging 2TB drives doing it now). The other 2 will be non redundant, no care for said *ISOs* and I plan to add 2-8 14-16TB drives. I'd love to stick with Debian, but at this point I can stick other setups on VMs and containers and be fine with that. Everything is currently in a SC933, but I'd like to go with something utilizing an ATX PSU instead of the triple redundant setup now (with UPS backing it up)

I just cant decide at this point which case I want to go with or which HDD setup I'd like, pool wise, to support such a bad habit as ISOs.
 
I would simply care about:
All data is important. Even movies or iso files as you need time and backups to restore them. Also according to Murphy a disaster will happen and this in a moment you need the files without time or backup to restore

Prefer a three step method against data loss
- (realtime) raid to be protected against a bad disk
- backup to be protected against a simple disaster (fire, theft)
- snap versioning to be protected against Ransomware or other incidents you will be aware days or weeks later
ex keep a snap per day/current week, a snap per week/current month, a snap per month/current year

A hot spare is a good method replace a bad disk immediatly. A higher level raid-Z is more worth. Think of it as a method with a hotspare already resilvered.

Regarding hardware
- use Sata/AHCI or LSI based HBAs and quality nics like Intel, prefer ECC and hotplug capable disk bays

https://napp-it.org/doc/downloads/zfs_design_principles.pdf
https://napp-it.org/doc/downloads/napp-it_build_examples.pdf
 
I have DNS CNAME aliases setup for our omniOS filer which is joined to AD. These have all worked great until we recently updated our domain and forest level to 2008 R2. Now we are unable to connect to the fileshares on the alias CNAMEs but can connect just fine to the actual hostname. Our problem is perfectly outlined here: https://docs.microsoft.com/en-us/tr...ame-alias-cannot-access-smb-file-server-share

The suggested fix is to use the netdom command which obviously doesn't work on omnios/solaris. And the "not reccomended" workaround is to use the setspn command to add an alias SPN for the filer. I have added the alias SPN for our server however the shares will still only connect from the actual filer's hostname. Any idea of a workaround for this? I'm assuming this is a fairly common issue with fileservers.
 
I have not seen problems but I am on a later Windows Server without a different alias.
To be clear. If hostname is omnis and cname is filer.domain.com

\\omnios ok
\\ip ok
\\filer not ok
\\filer.domain.com ?
 
I have not seen problems but I am on a later Windows Server without a different alias.
To be clear. If hostname is omnis and cname is filer.domain.com

\\omnios ok
\\ip ok
\\filer not ok
\\filer.domain.com ?
\\omnios ok
\\ip Not ok
\\filer not ok
\\filer.domain.com not ok

This was resolved by disabling signing in smb.conf on the filer:

Code:
signing_enabled=false
signing_required=false
 
  • Like
Reactions: _Gea
like this
To not irritating someone..

There is no smb.conf on the filer as this is not SAMBA but the multithreaded, kernelbased Solaris SMB server
where SMB server properties are either share, system or ZFS properties (ACL and locking). You list them via sharectl get smb
or edit in napp-it menu Services > SMB > Properties
 
Last edited:
OmniOS 151038q is out
https://github.com/omniosorg/omnios-build/blob/r151038/doc/ReleaseNotes.md
This is a security fix regarding OpenSSL bugs

btw.
OmniOS has a stable every 6 months with newest ZFS features and a long term stable every 2 years with mostly a freeze of new ZFS features but regular security and bug fixes under a release management with dedicated repositories. About twice a month there are security fixes like the current 151038q

All OmniOS releases are Opensource and free including the regular security and bug fixes. You can aquire a commercial support contract for OmniOS and as support with regular updates is a essential option for commercial storage use cases, Commercial Support

For private use you can support OmniOS as a patron, Donate
 
well, I have some logitech boxes that's why I need to use logitech media server
I have created an installer/package for the latest omnios version for logitech media server.
But it still needs a little bit of tweaking and testing.
I will publish it in the next days/weeks...
 
  • Like
Reactions: _Gea
like this
Hoping to get some help with a old ZFS filesystem please.

Importing 10 year old Pool from Nas4Free

Setting up a new filer server, running ESXI and Napp-it

Main issue: ZFS file system not showing up in Napp-it, but it is mounted and I can browse it from the command line. SKYPOOL/SKYNAS

I've got four old drives that came from Nas4Free, pool looks to be created over 10 years ago. I fired up a XigmaNAS to see if I could import the pool and it worked fine, but I still wanted to migrate over to Napp-it. Now I'm running into issues, more than likely how the old drives were formatted back in the day.

These are 4TB drives that I plan on swapping out with 8TB drive, taking them offline one at a time, let it resilver and do the rest. I then had a different idea to setup Napp-it on a diff machine and just copy it over with Syncoid.

1. I was going to copy all of the data over with Syncoid but it was running very slow, then I looked at the hardware and one of the drives failed, so I had to use a drive that I planned on using for the NAS upgrade to 8TB drives.
2. I ended up just swapping the 4TB drives one by one and letting them resilver to the 8TB drives. These 8TB drives originally had a file system on them from Napp-it (OmniOS) but I destroyed the filesystem I created on them as it was empty to begin with. My 4TB drives are the ones with the original information.
3. After all of the drives were finished being resilvered with XigmaNAS I shut down and fired up OmniOS. I could import the pool with Napp-it just fine, but the GUI does not show the ZFS file system as it normally does. Might be a permissions issue?
4. I also have an issue with the size of the file system as it should have close to doubled. The old file system before the drive swap was using 10.1TB

attaching a screenshot to show what I mean.

Also here is some output:

Code:
zfs list
NAME                                               USED  AVAIL  REFER  MOUNTPOINT
SKYPOOL                                           10.1T   432G  10.1T  /SKYPOOL/SKYNAS
SSDMirror                                         89.9G   899G    96K  /SSDMirror
rpool                                             9.33G  21.2G  25.5K  /rpool
rpool/ROOT                                        1.07G  21.2G    24K  /rpool/ROOT
rpool/ROOT/omnios-r151038k                        1.07G  21.2G   951M  legacy
rpool/ROOT/pre_download_19.12.homeuse_1626574172   131K  21.2G   925M  /
rpool/ROOT/pre_napp-it-18.12                       119K  21.2G   683M  /
rpool/dump                                        4.00G  21.2G  4.00G  -
rpool/export                                        48K  21.2G    24K  /export
rpool/export/home                                   24K  21.2G    24K  /export/home
rpool/swap                                        4.25G  25.4G  5.50M  -
root@omnios-gwar:/SKYPOOL/SKYNAS#


zpool status
  pool: SKYPOOL
 state: ONLINE
  scan: resilvered 3.37T in 0 days 14:59:47 with 0 errors on Mon Sep  6 20:24:07 2021
config:

        NAME                       STATE     READ WRITE CKSUM
        SKYPOOL                    ONLINE       0     0     0
          raidz1-0                 ONLINE       0     0     0
            c0t5000CCA252C83AD8d0  ONLINE       0     0     0
            c0t5000CCA26BD16F32d0  ONLINE       0     0     0
            c0t5000CCA266F7BC19d0  ONLINE       0     0     0
            c0t5000CCA26BC80AECd0  ONLINE       0     0     0
        logs
          c3t5CD2E45C0190B856d0    ONLINE       0     0     0

errors: No known data errors

  pool: SSDMirror
 state: ONLINE
  scan: none requested
config:

        NAME                        STATE     READ WRITE CKSUM
        SSDMirror                   ONLINE       0     0     0
          mirror-0                  ONLINE       0     0     0
            c10t002538D611420B5Fd0  ONLINE       0     0     0
            c9t002538D611420B7Bd0   ONLINE       0     0     0

errors: No known data errors

  pool: rpool
 state: ONLINE
  scan: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        rpool       ONLINE       0     0     0
          c1t0d0    ONLINE       0     0     0

errors: No known data errors
root@omnios-gwar:/SKYPOOL/SKYNAS#

Thanks for any help on this!
 

Attachments

  • zfs-FS.JPG
    zfs-FS.JPG
    54.4 KB · Views: 0
/SKYPOOL/SKYNAS is not a ZFS filesystem but only a mountpoint of SKYPOOL so napp-it cannot show.
I would set the mountpoint of SKYPOOL to /SKYPOOL to avoid irritations

NAME USED AVAIL REFER MOUNTPOINT
SKYPOOL 10.1T 432G 10.1T /SKYPOOL/SKYNAS
 
Gea --

Thanks for the reply. I exported my pool and reimported it, but first I had to remove the 'SKYNAS' folder from 'SKYPOOL', but still the filesystem doesn't show up in Napp-it as it normally does. I can open a SSH session and browse '/SKYPOOL' and I see my data. It is mounted for sure, but not sure why the GUI won't reflect that. Any ideas?

EDIT: I did create a new filesystem for the 'SSDMirror' as they were new drives. Also I've attached some pics of the SKYPOOL properties page.

Code:
zfs list
NAME                                               USED  AVAIL  REFER  MOUNTPOINT
SKYPOOL                                           10.1T   432G  10.1T  /SKYPOOL
SSDMirror                                         89.9G   899G    96K  /SSDMirror
SSDMirror/VM_AREA                                   96K   809G    96K  /SSDMirror/VM_AREA
rpool                                             9.33G  21.2G  25.5K  /rpool
rpool/ROOT                                        1.07G  21.2G    24K  /rpool/ROOT
rpool/ROOT/omnios-r151038k                        1.07G  21.2G   951M  legacy
rpool/ROOT/pre_download_19.12.homeuse_1626574172   131K  21.2G   925M  /
rpool/ROOT/pre_napp-it-18.12                       119K  21.2G   683M  /
rpool/dump                                        4.00G  21.2G  4.00G  -
rpool/export                                        48K  21.2G    24K  /export
rpool/export/home                                   24K  21.2G    24K  /export/home
rpool/swap                                        4.25G  25.4G  5.50M  -
root@omnios-gwar:/SKYPOOL#

1631140697372.png
 

Attachments

  • sk1prop.JPG
    sk1prop.JPG
    125.2 KB · Views: 1
  • sk2prop.JPG
    sk2prop.JPG
    26.7 KB · Views: 1
Last edited:
There is only the pool SKYPOOL, no filesytems below. If you want a filesystem skypool/skynas you must create it. The former "filesystem/folder" skypool/skynas was only a mountpoint
 
There is only the pool SKYPOOL, no filesytems below. If you want a filesystem skypool/skynas you must create it. The former "filesystem/folder" skypool/skynas was only a mountpoint
Prior to Nas4Free there was already a filesystem on the pool. If I create one in Napp-it, will it destroy what data is already there? I can browse the mount point /SKYPOOL but hoping I won't lose that data if I create a new filesystem. No, it doesn't have to be 'skypool/skynas'. I was just playing with mountpoints in hopes for Napp-it to recognize it.
 
All data of pool is stored in the pool, does not matter where you mount the pool so nothing to worry. If you create a new filesystem it will be empty. If there is already a filesystem with a given name, ZFS refuses to create the same again.

btw.
ZFS mountpoints are a user setting. I prefer /xx for a pool xx to avoid such confusions but ZFS GUIs are different for ex. some mount all pools under /volumes ex /volumes/xx.
 
All data of pool is stored in the pool, does not matter where you mount the pool so nothing to worry. If you create a new filesystem it will be empty. If there is already a filesystem with a given name, ZFS refuses to create the same again.

btw.
ZFS mountpoints are a user setting. I prefer /xx for a pool xx to avoid such confusions but ZFS GUIs are different for ex. some mount all pools under /volumes ex /volumes/xx.
Thx for the tip on the mounts.

When I create the new filesystem, since it'll be empty, how can I get my data into it?
 
When I create the new filesystem, since it'll be empty, how can I get my data into it?

- via Windows and an SMB share, best via RoboCopy (Windows sync tool, keeps ntfs alike ACL intact)
- via midnight commander (two window Unix console tool, start via mc, fast and simple local copy/move)
- via a napp-it data mover (mostly for repeated data move/copy actions)
- via rsync
- via console tools like cp or mv

I would use opt 2 as it is ultrafast. You can start mc remotely via a Putty session (as root to avoid permission problems)
 
Last edited:
Virtualisation options with OmniOS

With OmniOS you have mainly these options

1. ESXi
This is my favourite option and works below all VMs (Type-1)

Advantage:
Market leader
Best separation of all VMs including a storage VM. Fastest recovery after a crash.
Best performance also for non-Linux VMs. Storage Access via iSCSI, NFS or SMB (ZFS snaps as Windows previous versions)

Disadvantage
Network filesystem NFS not as fast as direct access

2. Linux LX Container

Advantage
Very low resource needs per Linux Container
Direkt access to ZFS filesystems

Disadvantage
You need Container ex from Joyent/SmartOS (alternative to ESXi, ProxMox).

3. KVM

4. Bhyve
I am under the impression that the main focus of SmartOS moves to Bhyve

VM management under OmniOS is done via zadm

Installation: pkg install zadm ->
/opt/ooce/zadm/bin/zadm

man: https://github.com/omniosorg/zadm/blob/master/doc/zadm.pod
With the help of the new zadm tool, you can install and manage the following VMs easily

Handling:
https://omnios.org/setup/zadm.html

or https://illumos.topicbox.com/groups/omnios-discuss (search for zadm, the new tool is under development by OmniOS)
I will move VM management in napp-it to zadm.
 
Last edited:
Ready to use 635 VMs in OmniOS (zadm list-images)


Code:
#zadm list-images
     PROVIDER  BRAND   NAME                                VERSION
 joyent    illumos Brocade vTM Developer               17.4.0
 joyent    illumos apache                              14.1.0
 joyent    illumos apache                              14.2.0
 joyent    illumos apache                              14.3.0
 joyent    illumos apache                              14.4.0
 joyent    illumos apache                              15.1.1
 joyent    illumos apache                              15.1.2
 joyent    illumos apache                              15.4.1
 joyent    illumos base                                13.1.0
 joyent    illumos base                                13.2.0
 joyent    illumos base                                13.2.1
 joyent    illumos base                                13.3.0
 joyent    illumos base                                13.3.1
 joyent    illumos base                                13.4.0
 joyent    illumos base                                13.4.1
 joyent    illumos base                                13.4.2
 joyent    illumos base                                14.1.0
 joyent    illumos base                                14.2.0
 joyent    illumos base                                14.3.0
 joyent    illumos base-32                             15.1.0
 joyent    illumos base-32                             15.1.1
 joyent    illumos base-32                             15.2.0
 joyent    illumos base-32                             15.3.0
 joyent    illumos base-32                             16.1.0
 joyent    illumos base-32                             16.2.0
 joyent    illumos base-32                             16.3.0
 joyent    illumos base-32                             16.3.1
 joyent    illumos base-32                             17.1.0
 joyent    illumos base-32                             17.2.0
 joyent    illumos base-32                             17.3.0
 joyent    illumos base-32                             18.1.0
 joyent    illumos base-32                             18.2.0
 joyent    illumos base-32                             18.3.0
 joyent    illumos base-32-lts                         14.4.0
 joyent    illumos base-32-lts                         14.4.1
 joyent    illumos base-32-lts                         14.4.2
 joyent    illumos base-32-lts                         15.4.0
 joyent    illumos base-32-lts                         15.4.1
 joyent    illumos base-32-lts                         16.4.1
 joyent    illumos base-32-lts                         17.4.0
 joyent    illumos base-32-lts                         18.4.0
 joyent    illumos base-64                             15.1.0
 joyent    illumos base-64                             15.1.1
 joyent    illumos base-64                             15.2.0
 joyent    illumos base-64                             15.3.0
 joyent    illumos base-64                             16.1.0
 joyent    illumos base-64                             16.2.0
 joyent    illumos base-64                             16.3.0
 joyent    illumos base-64                             16.3.1
 joyent    illumos base-64                             17.1.0
 joyent    illumos base-64                             17.2.0
 joyent    illumos base-64                             17.3.0
 joyent    illumos base-64                             18.1.0
 joyent    illumos base-64                             18.2.0
 joyent    illumos base-64                             18.3.0
 joyent    illumos base-64                             19.1.0
 joyent    illumos base-64                             19.2.0
 joyent    illumos base-64                             19.3.0
 joyent    illumos base-64                             20.1.0
 joyent    illumos base-64                             20.2.0
 joyent    illumos base-64                             20.3.0
 joyent    illumos base-64-lts                         14.4.0
 joyent    illumos base-64-lts                         14.4.1
 joyent    illumos base-64-lts                         14.4.2
 joyent    illumos base-64-lts                         15.4.0
 joyent    illumos base-64-lts                         15.4.1
 joyent    illumos base-64-lts                         16.4.1
 joyent    illumos base-64-lts                         17.4.0
 joyent    illumos base-64-lts                         18.4.0
 joyent    illumos base-64-lts                         19.4.0
 joyent    illumos base-64-lts                         20.4.0
 joyent    illumos base-64-trunk                       20200124
 joyent    illumos base-64-trunk                       20200420
 joyent    illumos base-64-trunk                       20201019
 joyent    illumos base-multiarch                      15.1.0
 joyent    illumos base-multiarch                      15.1.1
 joyent    illumos base-multiarch                      15.2.0
 joyent    illumos base-multiarch                      15.3.0
 joyent    illumos base-multiarch                      16.1.0
 joyent    illumos base-multiarch                      16.2.0
 joyent    illumos base-multiarch                      16.3.0
 joyent    illumos base-multiarch                      16.3.1
 joyent    illumos base-multiarch                      17.1.0
 joyent    illumos base-multiarch                      17.2.0
 joyent    illumos base-multiarch                      17.3.0
 joyent    illumos base-multiarch                      18.1.0
 joyent    illumos base-multiarch                      18.2.0
 joyent    illumos base-multiarch                      18.3.0
 joyent    illumos base-multiarch-lts                  14.4.0
 joyent    illumos base-multiarch-lts                  14.4.1
 joyent    illumos base-multiarch-lts                  14.4.2
 joyent    illumos base-multiarch-lts                  15.4.0
 joyent    illumos base-multiarch-lts                  15.4.1
 joyent    illumos base-multiarch-lts                  16.4.1
 joyent    illumos base-multiarch-lts                  17.4.0
 joyent    illumos base-multiarch-lts                  18.4.0
 joyent    illumos base64                              13.1.0
 joyent    illumos base64                              13.2.0
 joyent    illumos base64                              13.2.1
 joyent    illumos base64                              13.3.0
 joyent    illumos base64                              13.3.1
 joyent    illumos base64                              13.4.0
 joyent    illumos base64                              13.4.1
 joyent    illumos base64                              13.4.2
 joyent    illumos base64                              14.1.0
 joyent    illumos base64                              14.2.0
 joyent    illumos base64                              14.3.0
 joyent    illumos cassandra                           13.1.0
 joyent    illumos cassandra                           14.1.0
 joyent    illumos cassandra                           14.2.0
 joyent    illumos cassandra                           14.4.0
 joyent    illumos cassandra                           15.1.1
 joyent    illumos ghost                               13.4.2
 joyent    illumos ghost                               14.2.0
 joyent    illumos ghost                               14.3.0
 joyent    illumos ghost                               14.4.0
 joyent    illumos ghost                               15.1.1
 joyent    illumos ghost                               15.1.2
 joyent    illumos java                                13.1.0
 joyent    illumos java                                13.3.1
 joyent    illumos java                                14.1.0
 joyent    illumos java                                14.2.0
 joyent    illumos java                                14.3.0
 joyent    illumos java                                14.4.0
 joyent    illumos java                                15.1.1
 joyent    illumos mariadb                             15.1.1
 joyent    illumos minimal-32                          15.1.0
 joyent    illumos minimal-32                          15.1.1
 joyent    illumos minimal-32                          15.2.0
 joyent    illumos minimal-32                          15.3.0
 joyent    illumos minimal-32                          16.1.0
 joyent    illumos minimal-32                          16.2.0
 joyent    illumos minimal-32                          16.3.0
 joyent    illumos minimal-32                          16.3.1
 joyent    illumos minimal-32                          17.1.0
 joyent    illumos minimal-32                          17.2.0
 joyent    illumos minimal-32                          17.3.0
 joyent    illumos minimal-32                          18.1.0
 joyent    illumos minimal-32                          18.2.0
 joyent    illumos minimal-32                          18.3.0
 joyent    illumos minimal-32-lts                      14.4.0
 joyent    illumos minimal-32-lts                      14.4.1
 joyent    illumos minimal-32-lts                      14.4.2
 joyent    illumos minimal-32-lts                      15.4.0
 joyent    illumos minimal-32-lts                      15.4.1
 joyent    illumos minimal-32-lts                      16.4.1
 joyent    illumos minimal-32-lts                      17.4.0
 joyent    illumos minimal-32-lts                      18.4.0
 joyent    illumos minimal-64                          15.1.0
 joyent    illumos minimal-64                          15.1.1
 joyent    illumos minimal-64                          15.2.0
 joyent    illumos minimal-64                          15.3.0
 joyent    illumos minimal-64                          16.1.0
 joyent    illumos minimal-64                          16.2.0
 joyent    illumos minimal-64                          16.3.0
 joyent    illumos minimal-64                          16.3.1
 joyent    illumos minimal-64                          17.1.0
 joyent    illumos minimal-64                          17.2.0
 joyent    illumos minimal-64                          17.3.0
 joyent    illumos minimal-64                          18.1.0
 joyent    illumos minimal-64                          18.2.0
 joyent    illumos minimal-64                          18.3.0
 joyent    illumos minimal-64                          19.1.0
 joyent    illumos minimal-64                          19.2.0
 joyent    illumos minimal-64                          19.3.0
 joyent    illumos minimal-64                          20.1.0
 joyent    illumos minimal-64                          20.2.0
 joyent    illumos minimal-64                          20.3.0
 joyent    illumos minimal-64-lts                      14.4.0
 joyent    illumos minimal-64-lts                      14.4.1
 joyent    illumos minimal-64-lts                      14.4.2
 joyent    illumos minimal-64-lts                      15.4.0
 joyent    illumos minimal-64-lts                      15.4.1
 joyent    illumos minimal-64-lts                      16.4.1
 joyent    illumos minimal-64-lts                      17.4.0
 joyent    illumos minimal-64-lts                      18.4.0
 joyent    illumos minimal-64-lts                      19.4.0
 joyent    illumos minimal-64-lts                      20.4.0
 joyent    illumos minimal-64-trunk                    20200124
 joyent    illumos minimal-64-trunk                    20200420
 joyent    illumos minimal-64-trunk                    20201019
 joyent    illumos minimal-multiarch                   15.1.0
 joyent    illumos minimal-multiarch                   15.1.1
 joyent    illumos minimal-multiarch                   15.2.0
 joyent    illumos minimal-multiarch                   15.3.0
 joyent    illumos minimal-multiarch                   16.1.0
 joyent    illumos minimal-multiarch                   16.2.0
 joyent    illumos minimal-multiarch                   16.3.0
 joyent    illumos minimal-multiarch                   16.3.1
 joyent    illumos minimal-multiarch                   17.1.0
 joyent    illumos minimal-multiarch                   17.2.0
 joyent    illumos minimal-multiarch                   17.3.0
 joyent    illumos minimal-multiarch                   18.1.0
 joyent    illumos minimal-multiarch                   18.2.0
 joyent    illumos minimal-multiarch                   18.3.0
 joyent    illumos minimal-multiarch-lts               14.4.0
 joyent    illumos minimal-multiarch-lts               14.4.1
 joyent    illumos minimal-multiarch-lts               14.4.2
 joyent    illumos minimal-multiarch-lts               15.4.0
 joyent    illumos minimal-multiarch-lts               15.4.1
 joyent    illumos minimal-multiarch-lts               16.4.1
 joyent    illumos minimal-multiarch-lts               17.4.0
 joyent    illumos minimal-multiarch-lts               18.4.0
 joyent    illumos mongodb                             13.2.0
 joyent    illumos mongodb                             13.2.1
 joyent    illumos mongodb                             13.3.1
 joyent    illumos mongodb                             14.1.0
 joyent    illumos mongodb                             14.1.1
 joyent    illumos mongodb                             14.2.0
 joyent    illumos mongodb                             14.3.0
 joyent    illumos mongodb                             14.4.0
 joyent    illumos mongodb                             15.1.1
 joyent    illumos mongodb                             15.4.1
 joyent    illumos mongodb                             16.4.1
 joyent    illumos multiarch                           13.3.0
 joyent    illumos multiarch                           13.3.1
 joyent    illumos multiarch                           14.3.0
 joyent    illumos mysql-cluster                       14.1.0
 joyent    illumos mysql-cluster                       14.2.0
 joyent    illumos mysql-cluster                       14.3.0
 joyent    illumos mysql-cluster                       14.4.0
 joyent    illumos mysql-cluster                       15.1.1
 joyent    illumos nginx                               14.1.0
 joyent    illumos nginx                               14.2.0
 joyent    illumos nginx                               14.3.0
 joyent    illumos nginx                               14.4.0
 joyent    illumos nginx                               15.1.1
 joyent    illumos nginx                               15.4.1
 joyent    illumos nodejs                              13.1.0
 joyent    illumos nodejs                              13.2.1
 joyent    illumos nodejs                              13.2.2
 joyent    illumos nodejs                              13.3.1
 joyent    illumos nodejs                              14.1.0
 joyent    illumos nodejs                              14.1.1
 joyent    illumos nodejs                              14.2.0
 joyent    illumos nodejs                              14.3.0
 joyent    illumos nodejs                              14.4.0
 joyent    illumos nodejs                              15.1.1
 joyent    illumos nodejs                              15.1.2
 joyent    illumos nodejs                              14.4.1
 joyent    illumos nodejs                              15.4.1
 joyent    illumos nodejs                              16.4.1
 joyent    illumos percona                             13.1.0
 joyent    illumos percona                             13.3.0
 joyent    illumos percona                             13.4.2
 joyent    illumos percona                             14.2.0
 joyent    illumos percona                             14.3.0
 joyent    illumos percona                             14.4.0
 joyent    illumos percona                             15.1.1
 joyent    illumos percona                             15.4.1
 joyent    illumos percona                             14.4.2
 joyent    illumos percona                             16.4.1
 joyent    illumos percona-cluster                     14.3.0
 joyent    illumos percona-cluster                     14.4.0
 joyent    illumos percona-cluster                     15.1.1
 joyent    illumos pkgbuild                            15.4.0
 joyent    illumos pkgbuild                            15.4.1
 joyent    illumos pkgbuild                            16.1.0
 joyent    illumos pkgbuild                            16.2.0
 joyent    illumos pkgbuild                            16.3.0
 joyent    illumos pkgbuild                            16.3.1
 joyent    illumos pkgbuild                            16.4.1
 joyent    illumos pkgbuild                            17.1.0
 joyent    illumos pkgbuild                            17.2.0
 joyent    illumos pkgbuild                            17.3.0
 joyent    illumos pkgbuild                            17.4.0
 joyent    illumos pkgbuild                            18.1.0
 joyent    illumos pkgbuild                            18.2.0
 joyent    illumos pkgbuild                            18.3.0
 joyent    illumos pkgbuild                            18.4.0
 joyent    illumos pkgbuild                            19.1.0
 joyent    illumos pkgbuild                            19.2.0
 joyent    illumos pkgbuild                            19.3.0
 joyent    illumos pkgbuild                            19.4.0
 joyent    illumos pkgbuild                            20.1.0
 joyent    illumos pkgbuild                            20.2.0
 joyent    illumos pkgbuild                            20.3.0
 joyent    illumos pkgbuild-lts                        20.4.0
 joyent    illumos pkgbuild-trunk                      20200124
 joyent    illumos pkgbuild-trunk                      20200420
 joyent    illumos pkgbuild-trunk                      20201019
 joyent    illumos postgresql                          13.3.1
 joyent    illumos postgresql                          14.1.0
 joyent    illumos postgresql                          14.2.0
 joyent    illumos postgresql                          14.2.1
 joyent    illumos postgresql                          14.3.0
 joyent    illumos postgresql                          14.4.0
 joyent    illumos postgresql                          15.1.1
 joyent    illumos postgresql                          15.4.1
 joyent    illumos postgresql                          16.4.1
 joyent    illumos redis                               14.2.0
 joyent    illumos redis                               14.3.0
 joyent    illumos redis                               14.4.0
 joyent    illumos redis                               15.1.1
 joyent    illumos standard                            13.2.0
 joyent    illumos standard                            13.2.1
 joyent    illumos standard                            13.2.3
 joyent    illumos standard                            13.3.1
 joyent    illumos standard                            13.4.0
 joyent    illumos standard                            13.4.1
 joyent    illumos standard                            14.1.0
 joyent    illumos standard                            14.1.1
 joyent    illumos standard                            14.2.0
 joyent    illumos standard                            14.2.1
 joyent    illumos standard                            14.3.0
 joyent    illumos standard                            14.4.0
 joyent    illumos standard                            15.1.1
 joyent    illumos standard64                          13.2.0
 joyent    illumos standard64                          13.2.1
 joyent    illumos standard64                          13.2.3
 joyent    illumos standard64                          13.3.1
 joyent    illumos standard64                          13.4.0
 joyent    illumos standard64                          13.4.1
 joyent    illumos standard64                          14.1.0
 joyent    illumos standard64                          14.1.1
 joyent    illumos standard64                          14.2.0
 joyent    illumos standard64                          14.2.1
 joyent    illumos standard64                          14.3.0
 joyent    illumos standard64                          14.4.0
 joyent    illumos steelapp-developer                  14.2.0
 joyent    illumos steelapp-developer                  14.3.0
 joyent    illumos steelapp-developer                  14.4.0
 joyent    illumos steelapp-developer                  15.1.1
 joyent    illumos steelapp-developer                  15.4.0
 joyent    illumos steelapp-developer                  15.4.1
 joyent    illumos stm-developer                       13.2.0
 joyent    illumos stm-developer                       13.4.1
 joyent    illumos triton-origin-multiarch-15.4.1      1.0.1
 joyent    illumos triton-origin-multiarch-18.1.0      1.0.1
 joyent    kvm     centos-6                            20140710
 joyent    kvm     centos-6                            20140828
 joyent    kvm     centos-6                            20141001
 joyent    kvm     centos-6                            20141112
 joyent    kvm     centos-6                            20141202
 joyent    kvm     centos-6                            20150128
 joyent    kvm     centos-6                            20150324
 joyent    kvm     centos-6                            20150630
 joyent    kvm     centos-6                            20150811
 joyent    kvm     centos-6                            20160111
 joyent    kvm     centos-6                            20161028
 joyent    kvm     centos-6                            20170327
 joyent    kvm     centos-6                            20170407
 joyent    kvm     centos-6                            20170621
 joyent    kvm     centos-7                            20140820
 joyent    kvm     centos-7                            20141001
 joyent    kvm     centos-7                            20141112
 joyent    kvm     centos-7                            20141202
 joyent    kvm     centos-7                            20150128
 joyent    kvm     centos-7                            20150324
 joyent    kvm     centos-7                            20150406
 joyent    kvm     centos-7                            20150630
 joyent    kvm     centos-7                            20160415
 joyent    kvm     centos-7                            20161028
 joyent    kvm     centos-7                            20170327
 joyent    kvm     centos-7                            20180320
 joyent    kvm     centos-7                            20180819
 joyent    kvm     debian-7                            20140709
 joyent    kvm     debian-7                            20140729
 joyent    kvm     debian-7                            20141001
 joyent    kvm     debian-7                            20141202
 joyent    kvm     debian-7                            20150119
 joyent    kvm     debian-7                            20150128
 joyent    kvm     debian-7                            20150325
 joyent    kvm     debian-7                            20150702
 joyent    kvm     debian-7                            20161025
 joyent    kvm     debian-8                            20150527
 joyent    kvm     debian-8                            20150702
 joyent    kvm     debian-8                            20161025
 joyent    kvm     debian-8                            20170410
 joyent    kvm     fedora-22                           20151015
 joyent    kvm     fedora-25                           20170206
 joyent    kvm     fedora-25                           20170328
 joyent    kvm     freebsd-10                          20160415
 joyent    kvm     freebsd-10                          20170403
 joyent    kvm     freebsd-11                          20180213
 joyent    kvm     freebsd-12                          20190424
 joyent    kvm     openbsd-6                           20170523
 joyent    kvm     ubuntu-16.04-rancheragent           1.0.0
 joyent    kvm     ubuntu-16.04-rancherserver          1.0.0
 joyent    kvm     ubuntu-certified-14.04              20140929
..
 joyent    kvm     ubuntu-certified-18.04              20180808
 joyent    lx      alpine-3                            20151007
..
 joyent    lx      centos-6                            20150618
..
 joyent    lx      centos-7                            20180323
 joyent    lx      couchbase-4                         20160411
 joyent    lx      debian-7                            20150618
..
 joyent    lx      debian-7                            20161213
 joyent    lx      debian-8                            20151109
..
 joyent    lx      debian-8                            20170214
 joyent    lx      debian-9                            20180404
 joyent    lx      lx-centos-6                         20150313
 joyent    lx      lx-centos-6                         20150320
 joyent    lx      lx-centos-6                         20150415
 joyent    lx      lx-centos-6                         20150601
 joyent    lx      lx-debian-7                         20150320
 joyent    lx      lx-debian-7                         20150415
 joyent    lx      lx-debian-7                         20150601
 joyent    lx      lx-ubuntu-14.04                     20150320
 joyent    lx      lx-ubuntu-14.04                     20150415
 joyent    lx      lx-ubuntu-14.04                     20150601
 joyent    lx      mongodb-3                           20160218
 joyent    lx      mongodb-3                           20170405
 joyent    lx      nginx-1                             20160413
 joyent    lx      nginx-1                             20170404
 joyent    lx      nodejs-4                            20160224
..
 joyent    lx      nodejs-4                            20170404
 joyent    lx      nodejs-6                            20161125
 joyent    lx      nodejs-6                            20170206
 joyent    lx      nodejs-6                            20170227
 joyent    lx      nodejs-6                            20170328
 joyent    lx      nodejs-6                            20170404
 joyent    lx      percona-5                           20160218
 joyent    lx      percona-5                           20170405
 joyent    lx      postgresql-9                        20160218
 joyent    lx      postgresql-9                        20170405
 joyent    lx      ubuntu-14.04                        20150618
..
 joyent    lx      ubuntu-14.04                        20170403
 joyent    lx      ubuntu-16.04                        20160427
 ..
 joyent    lx      ubuntu-16.04                        20170403
 joyent    lx      ubuntu-20.04                        20210413
 joyent    lx      void                                20201016
 joyent    lx      void                                20210413
 omnios    bhyve   omnios-bloody-20210819              bloody-20210819
 omnios    bhyve   omnios-r151034l                     r151034l
 omnios    bhyve   omnios-r151036                      r151036
 omnios    bhyve   omnios-r151036m                     r151036m
 omnios    bhyve   omnios-r151038                      r151038
 omnios    bhyve   omnios-r151038k                     r151038k
 omnios    bhyve   omnios-r151038v                     r151038v
 omnios    illumos omnios-bloody-20210819              bloody-20210819
 omnios    illumos omnios-r151030ap                    r151030ap
 omnios    illumos omnios-r151032p                     r151032p
 omnios    illumos omnios-r151034l                     r151034l
 omnios    illumos omnios-r151036                      r151036
 omnios    illumos omnios-r151036m                     r151036m
 omnios    illumos omnios-r151038                      r151038
 omnios    illumos omnios-r151038k                     r151038k
 omnios    illumos omnios-r151038v                     r151038v
 omnios    lx      lx-debian-buster-2020-10-05         debian-buster
 omnios    lx      lx-debian-buster-2021-03-02         debian-buster
 omnios    lx      lx-debian-buster-2021-03-30         debian-buster
 omnios    lx      lx-devuan-beowulf-2020-10-05        devuan-beowulf
 omnios    lx      lx-devuan-beowulf-2021-03-02        devuan-beowulf
 omnios    lx      lx-devuan-beowulf-2021-03-30        devuan-beowulf
 omnios    lx      lx-ubuntu-18.04-2020-10-05          ubuntu-18.04
 omnios    lx      lx-ubuntu-18.04-2021-03-02          ubuntu-18.04
 omnios    lx      lx-ubuntu-18.04-2021-03-30          ubuntu-18.04
 omnios    lx      lx-ubuntu-20.04-2020-10-05          ubuntu-20.04
 omnios    lx      lx-ubuntu-20.04-2021-03-02          ubuntu-20.04
 omnios    lx      lx-ubuntu-20.04-2021-03-30          ubuntu-20.04
 omnios    lx      lx-void-2020-10-05                  void-2020
 omnios    lx      lx-void-2021-03-02                  void-2021
 omnios    lx      lx-void-2021-03-30                  void-2021
 omnios-us bhyve   omnios-bloody-20210819              bloody-20210819
 omnios-us bhyve   omnios-r151034l                     r151034l
 omnios-us bhyve   omnios-r151036                      r151036
 omnios-us bhyve   omnios-r151036m                     r151036m
 omnios-us bhyve   omnios-r151038                      r151038
 omnios-us bhyve   omnios-r151038k                     r151038k
 omnios-us bhyve   omnios-r151038v                     r151038v
 omnios-us illumos omnios-bloody-20210819              bloody-20210819
 omnios-us illumos omnios-r151030ap                    r151030ap
..
 omnios-us illumos omnios-r151038v                     r151038v
 omnios-us lx      lx-debian-buster-2020-10-05         debian-buster
 omnios-us lx      lx-debian-buster-2021-03-02         debian-buster
 omnios-us lx      lx-debian-buster-2021-03-30         debian-buster
 omnios-us lx      lx-devuan-beowulf-2020-10-05        devuan-beowulf
 omnios-us lx      lx-devuan-beowulf-2021-03-02        devuan-beowulf
 omnios-us lx      lx-devuan-beowulf-2021-03-30        devuan-beowulf
 omnios-us lx      lx-ubuntu-18.04-2020-10-05          ubuntu-18.04
 omnios-us lx      lx-ubuntu-18.04-2021-03-02          ubuntu-18.04
 omnios-us lx      lx-ubuntu-18.04-2021-03-30          ubuntu-18.04
 omnios-us lx      lx-ubuntu-20.04-2020-10-05          ubuntu-20.04
 omnios-us lx      lx-ubuntu-20.04-2021-03-02          ubuntu-20.04
 omnios-us lx      lx-ubuntu-20.04-2021-03-30          ubuntu-20.04
 omnios-us lx      lx-void-2020-10-05                  void-2020
 omnios-us lx      lx-void-2021-03-02                  void-2021
 omnios-us lx      lx-void-2021-03-30                  void-2021
 proxmox   lx      almalinux-8-default                 20210928
 proxmox   lx      alpine-3.12-default                 20200823
 proxmox   lx      alpine-3.13-default                 20210419
 proxmox   lx      alpine-3.14-default                 20210623
 proxmox   lx      archlinux-base                      20210420-1
 proxmox   lx      centos-7-default                    20190926
 proxmox   lx      centos-8-default                    20201210
 proxmox   lx      debian-10-standard                  10.7-1
 proxmox   lx      debian-11-standard                  11.0-1
 proxmox   lx      devuan-3.0-standard                 3.0
 proxmox   lx      fedora-33-default                   20201115
 proxmox   lx      fedora-34-default                   20210427
 proxmox   lx      gentoo-current-default              20200310
 proxmox   lx      opensuse-15.3-default               20210925
 proxmox   lx      proxmox-mailgateway-6.4-standard    6.4-1
 proxmox   lx      proxmox-mailgateway-7.0-standard    7.0-1
 proxmox   lx      rockylinux-8-default                20210929
 proxmox   lx      ubuntu-16.04-standard               16.04.5-1
 proxmox   lx      ubuntu-18.04-standard               18.04.1-1
 proxmox   lx      ubuntu-20.04-standard               20.04-1
 proxmox   lx      ubuntu-21.04-standard               21.04-1
 
SAS high performance storage (2 x 12G multipath)

High performance, high capacity, high number of disks and hotplug capability are quite common demands for storage. For high performance the first thought is NVMe. But when it comes to number of disks say dozens or hundreds or hotplug capability you may consider 2x12G Multipath SAS or 24G SAS in future.

Quite common choices for SAS disks are the affordable Samsung 1643, the WD SS 530 (quite the best SAS SSD, but EoL with the newer SS540 hard to find) and the Seagate Nytro 3732 as an alternative.

see my benchmarks

http://napp-it.org/doc/downloads/sas.pdf
 
For those who are using mpio SAS (due better performance or in a HA cluster):

There is a bug in mpathadm in OmniOS 151038lts (the multipath admin tool).
A fix is available, Topicbox
 
After a new installation of OmniOS 151040, you need the following links for napp-it (or a rerun of the wget installer)

ln -s /lib/libssl.so /usr/lib/libssl.so.1.0.0
ln -s /lib/libcrypto.so /usr/lib/libcrypto.so.1.0.0
 
Last edited:
new in napp-it 21.dev

Push alerts via Pushover, Telegram, SendinBlue or your own api:
Per default a push use the following webapi
/var/web-gui/data/napp-it/zfsos/_lib/scripts/webapi.pl

If there is a my file
/var/web-gui/_my/scripts/webapi/webapi.pl it is used instead (update save)
 
Update for minIO users:
napp-it 20.dev from nov 09 supports the the new minIO settings (required for minIO newer may 2021)

- ROOT_USER and ROOT_PASSWORD instead the former KEY and SECRET
- a new webconsole at port 800x (1000 lower than service port) with support for user, groups and permissions
- /var/web-gui/_log/minio/certs is new cert directory
 
Back
Top