OpenSolaris derived ZFS NAS/ SAN (OmniOS, OpenIndiana, Solaris and napp-it)

_Gea

2[H]4U
Joined
Dec 5, 2010
Messages
4,008
Ever since I went to the latest stable omniOS I am getting weird file modified behavior. If I make a VBS file in notepad++ and keep it open while I execute the script, I get a prompt saying the file has been modified. It used to not modify the file after I executed it. Any idea what's going on here?
is atime disabled?
 

docjay

n00b
Joined
Dec 1, 2014
Messages
7
OmniOS v151038
Napp-it v19.12

Gea,

I'm updating my hardware to a Dell R720 with the 3.5 bays up front. Looking to get your opinion on drives and how they are setup for the SLOG, OmniOS, and the VM storage pieces please.

New server specs: Dell R720
Memory: 96GB RAM
CPU: (2) Intel Xeon E5-2600 series
Dell H710P flashed to IT Mode
(4) 8TB WDC SATA for main large pool

other parts lying around
(1) Samsung 983 DCT U.2 SSD (within my budget)
(1) WD-Black SN750 NVMe SSD
(1) Intel Optane M.2 80mm PCIe 3.0 32GB SSD
(also can use the SSDs in my current setup)

I'm currently running ESXi 6.5 as my boot OS and have a separate SSD drive presented to ESXI for OmniOS. Also I have a mirror with two SSDs for my VM area that is running out of space.

The zpool status below is my setup currently running on an aging Supermicro board. I plan on moving over the raidz1-0 pool and backup the OmniOS VM to move over to my new hardware.

Plan so far for new hardware setup:
1. Install ESXI 7.x on a USB Flash drive
2. Move over the current raidz1-0 pool
3. backup the OmniOS VM and reuse
4. Use the Samsung 983 DCT for the SLOG

Questions:
1) Should I use a separate SSD just for OmniOS?
2) Is it a good idea to create a pool just for where my VMs will live or should these run on faster drives than just the 7200RPM SATA disks that are in the raidz1 pool?
3) have I missed anything, or can you provide any suggestions?

Thank you so much for all of help you've given over the years - greatly appreciated!


Code:
zpool status  

  pool: NAPPTANK
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
    still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
    the pool may no longer be accessible by software that does not support
    the features. See zpool-features(5) for details.
  scan: scrub repaired 0 in 1 days 07:49:09 with 0 errors on Tue Jul  6 06:49:12 2021
config:

    NAME                       STATE     READ WRITE CKSUM      CAP            Product /napp-it   SN/LUN           IOstat mess       SMART
    NAPPTANK                   ONLINE       0     0     0
      raidz1-0                 ONLINE       0     0     0
        c3t5000CCA26BD16F32d0  ONLINE       0     0     0      8 TB           WDC WD80EMAZ-00W   S:0 H:0 T:0       -
        c3t5000CCA266F7BC19d0  ONLINE       0     0     0      8 TB           WDC WD80EMAZ-00W   S:0 H:0 T:0       -
        c3t5000CCA252C83AD8d0  ONLINE       0     0     0      8 TB           WDC WD80EMAZ-00W   S:0 H:0 T:0       -
        c3t5000CCA26BC80AECd0  ONLINE       0     0     0      8 TB           WDC WD80EMAZ-00W   S:0 H:0 T:0       -
    logs  
      c3t55CD2E404B6E030Fd0    ONLINE       0     0     0      100 GB         INTEL SSDSC2BA10   S:0 H:0 T:0       -

errors: No known data errors

  pool: SSD_MIRROR
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
    still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
    the pool may no longer be accessible by software that does not support
    the features. See zpool-features(5) for details.
  scan: scrub repaired 0 in 0 days 00:12:43 with 0 errors on Sun Jul  4 23:12:46 2021
config:

    NAME                       STATE     READ WRITE CKSUM      CAP            Product /napp-it   SN/LUN           IOstat mess       SMART
    SSD_MIRROR                 ONLINE       0     0     0
      mirror-0                 ONLINE       0     0     0
        c3t50025388A07E6676d0  ONLINE       0     0     0      512.1 GB       Samsung SSD 850    S:0 H:0 T:0       -
        c3t5002538840224D6Dd0  ONLINE       0     0     0      512.1 GB       Samsung SSD 850    S:0 H:0 T:0       -

errors: No known data errors

  pool: rpool
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
    still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
    the pool may no longer be accessible by software that does not support
    the features. See zpool-features(5) for details.
  scan: none requested
config:

    NAME        STATE     READ WRITE CKSUM      CAP            Product /napp-it   SN/LUN           IOstat mess       SMART
    rpool       ONLINE       0     0     0
      c2t0d0s0  ONLINE       0     0     0      32.2 GB        Virtual disk       S:0 H:0 T:0       -

errors: No known data errors

Code:
current disks:

 id                              part       identify       stat       diskcap       partcap       error       vendor       product       sn       temp       smart_overview2
 c1t0d0       -       -       removed               -        S:0 H:5 T:0       NECVMWar       VMware IDE CDR10               -       -
 c2t0d0       single       via dd        ok               32.2 GB        S:0 H:0 T:0       VMware       Virtual disk              -       -
 c3t5000CCA252C83AD8d0       single       via dd        ok               8 TB        S:0 H:0 T:0       ATA       WDC WD80EMAZ-00W              -       -
 c3t5000CCA266F7BC19d0       single       via dd        ok               8 TB        S:0 H:0 T:0       ATA       WDC WD80EMAZ-00W             -       -
 c3t5000CCA26BC80AECd0       single       via dd        ok               8 TB        S:0 H:3 T:2       ATA       WDC WD80EMAZ-00W             -       -
 c3t5000CCA26BD16F32d0       single       via dd        ok               8 TB        S:0 H:0 T:0       ATA       WDC WD80EMAZ-00W              -       -
 c3t5002538840224D6Dd0       single       via dd        ok               512.1 GB        S:0 H:0 T:0       ATA       Samsung SSD 850              -       -
 c3t50025388A07E6676d0       single       via dd        ok               512.1 GB        S:0 H:0 T:0       ATA       Samsung SSD 850            -       -
 c3t55CD2E404B6E030Fd0       single       via dd        ok               100 GB        S:0 H:0 T:0       ATA       INTEL SSDSC2BA10
 
Last edited:

_Gea

2[H]4U
Joined
Dec 5, 2010
Messages
4,008
OmniOS v151038
Napp-it v19.12

Gea,

I'm updating my hardware to a Dell R720 with the 3.5 bays up front. Looking to get your opinion on drives and how they are setup for the SLOG, OmniOS, and the VM storage pieces please.

New server specs: Dell R720
Memory: 96GB RAM
CPU: (2) Intel Xeon E5-2600 series
Dell H710P flashed to IT Mode
(4) 8TB WDC SATA for main large pool

other parts lying around
(1) Samsung 983 DCT U.2 SSD (within my budget)
(1) WD-Black SN750 NVMe SSD
(1) Intel Optane M.2 80mm PCIe 3.0 32GB SSD
(also can use the SSDs in my current setup)

I'm currently running ESXi 6.5 as my boot OS and have a separate SSD drive presented to ESXI for OmniOS. Also I have a mirror with two SSDs for my VM area that is running out of space.

The zpool status below is my setup currently running on an aging Supermicro board. I plan on moving over the raidz1-0 pool and backup the OmniOS VM to move over to my new hardware.

Plan so far for new hardware setup:
1. Install ESXI 7.x on a USB Flash drive
2. Move over the current raidz1-0 pool
3. backup the OmniOS VM and reuse
should work
4. Use the Samsung 983 DCT for the SLOG
should be slightly faster than the curent DC 3700
Questions:
1) Should I use a separate SSD just for OmniOS?
You need a bootdrive for ESXi (USB or Sata) and a local datastore for OmniOS. I usually use an Sata SSD (60GB+) for both - ex the DC 3700
2) Is it a good idea to create a pool just for where my VMs will live or should these run on faster drives than just the 7200RPM SATA disks that are in the raidz1 pool?
Common is a large disk based pool for Filer/Backup and a second faster SSD based pool for VMs.
3) have I missed anything, or can you provide any suggestions?

Thank you so much for all of help you've given over the years - greatly appreciated!
thanks, you are welcome
 

chune

Weaksauce
Joined
Nov 2, 2013
Messages
68
is atime disabled?
It wasn't but I disabled it and the issue persisted. Finally figured it out today. Turns out the napp-it vm was 4 mins ahead of time so everything was just constantly thinking it was modified but from the future. This napp-it vm is domain joined so I thought it would NTP sync automatically with the DC but I guess that isn't the case. I manually set the time to be correct and its working now.
 

chune

Weaksauce
Joined
Nov 2, 2013
Messages
68
Completely unrelated, but is there a way to configure file access audits in napp-it? I know solaris can do it but not sure if omnios can
 

arryo

n00b
Joined
May 23, 2012
Messages
54
I'm currently using ESXI 6.5 with Napp-it + Omni as VM, should I upgrade to ESXI 6.7?

Thanks
 

_Gea

2[H]4U
Joined
Dec 5, 2010
Messages
4,008
New default napp-it is v 21.06 for all releases incl. free
Free is ok for commercial use and without a capacity limit but without support or updates to dev/support releases or PRO addons
 

pyr02k1

Limp Gawd
Joined
Nov 29, 2010
Messages
423
_Gea Could I get an opinion from you? How would you handle less important *ISOs* mixed with personal photos and such, and some CPU for converting said *ISOs* around while supporting a few game servers?
I have 8x8TB drives right now in a mergerfs and I'm irritated by it more than anything. My important stuff sits on a Raid Z2 with some aging and close to full drives but that's less of an issue. I'm planning to replace my aging 2650v2 setup with something more modern, like a 5800x with some ECC, but I'm not terribly picky. Trying to cut back my idle 300w usage to something more reasonable while adding some capability to the system. I'd rather not spend over 1k on this upgrade (not including HDDs), but it's flexible.

If you had to build your ideal server, with or without hot swap (and hot spares), what would you shoot for today? My goal is to keep the 8x8TB in use, splitting 5 into a Raid Z2 with a hot spare on top for important data (to replace the aging 2TB drives doing it now). The other 2 will be non redundant, no care for said *ISOs* and I plan to add 2-8 14-16TB drives. I'd love to stick with Debian, but at this point I can stick other setups on VMs and containers and be fine with that. Everything is currently in a SC933, but I'd like to go with something utilizing an ATX PSU instead of the triple redundant setup now (with UPS backing it up)

I just cant decide at this point which case I want to go with or which HDD setup I'd like, pool wise, to support such a bad habit as ISOs.
 
Top