OpenSolaris derived ZFS NAS/ SAN (OmniOS, OpenIndiana, Solaris and napp-it)

I am not aware of problems with Hipster but to be honest I do not use HD shutdown.
One thing you need to ensure is that there is no service accessing the disk regularly like a napp-it alert job or the Solaris fault management daemon fmd that you may need to disable
 
I have a RAIDZ2 comprised of 8x 3TB HGST 512 sector drives that have over 6 years of power on time (approx 53200hrs). While I haven't lost a drive yet, I know my days are numbered. The pool was created using ashift=9 and and I know that in the past OI/OmniOS wouldn't let you replace the drive with a modern 4k drive if the pool was configured this way. Do those limitations still exist or do I need to re-create my pool with ashift=12 so that I freely replace my drives when they start to kick the bucket?
 
Yes this problem persists (I suppose on any OpenZFS/ZFS).
And if you can get it working with a 512e disk, this would result in a performance degration

The only proper way is to recreate the pool with ashift=12. With 512n disks this happens when you mix a physical 4k disk in a vdev or if you set this in sd.conf
 
Running OmniOS + Napp-IT 17.01

Been working great so far, but just added 10 new disks (SAS disks connected via LSI 2007 HBA)

OS seems fine, ZFS pools still working, but in Napp-IT, if I click on disks or pools, browser just spins.

minilog
--- _lib _lib/illumos/get-disk.pl &get_mydisk_format 20 <- admin.pl (eval) 833 <- admin.pl &load_lib 515 <- admin.pl &get_preloads 272 ---

exe(get-disk.pl 86): format

I can root the box and do an iostat or zpool list and it works fine.

Rebooting would be a PITA. any tips on where to check/look?
 
Running OmniOS + Napp-IT 17.01

Been working great so far, but just added 10 new disks (SAS disks connected via LSI 2007 HBA)

OS seems fine, ZFS pools still working, but in Napp-IT, if I click on disks or pools, browser just spins.

minilog
--- _lib _lib/illumos/get-disk.pl &get_mydisk_format 20 <- admin.pl (eval) 833 <- admin.pl &load_lib 515 <- admin.pl &get_preloads 272 ---

exe(get-disk.pl 86): format

I can root the box and do an iostat or zpool list and it works fine.

Rebooting would be a PITA. any tips on where to check/look?

Looks like i spoke too soon. I let the browser spin and spin, wrote the above, came back 5 mins later and it came up. Now when I hit pools or disks it comes up reasonably fast.

Not sure if this has anything to do with it, fairly certain one of the newly added disks is bad (*shakes fist!!*)

Best to update napp-it while I am here lurking?
 
It seems that the format command is hanging as this is executed by napp-it to detect connected disks. Try the format command and a zpool list at console to check behaviour. The format command can be ended after listing with ctrl-c

If format or zpool status hang, remove the new disks and insert disk by disk, wait a little and check if format is listing the disk to find a bad disk.

If format is listing all disks but stops at a special disk for a while remove it.
Update of napp-it to current is an option but would not help as this seems a disk problem.
 
Gea,

I do notice that as of 151022, when upgrading napp-it, the installation is broken until another login. (This actually took me a while to realize that it could be fixed with just a login, and I had been rolling back to a pre-update BE on my test machine to await a working napp-it version).

This breakage is most inconvenient because root's run of auto.pl certainly does not wait for another login, so it is broken until then. Do you think it napp-it might run the fix-up that happens during login w.r.t. OS-specific packages (e.g., /tools/omni_022/CGI/) just after the update so as not to break auto.pl?

Thank you!
 
I'm adding another disk to Napp-it (passed through via ESXi) just for snapshots. I'm wondering if there is a way to tell Napp-IT to create snapshots onto my new disk?

Also, it looks like there is a way to manage snapshots of my VMs through the napp-it interface? Jobs --> ESXI Hot-snaps

Thanks for any help with this
 
You'll need to create a new pool as the destination for zfs send/recv from your live pool. Older snaps can be removed from the live pool, but it must retain the last sent so that send/recv can work.
 
Gea,

I do notice that as of 151022, when upgrading napp-it, the installation is broken until another login. (This actually took me a while to realize that it could be fixed with just a login, and I had been rolling back to a pre-update BE on my test machine to await a working napp-it version).

This breakage is most inconvenient because root's run of auto.pl certainly does not wait for another login, so it is broken until then. Do you think it napp-it might run the fix-up that happens during login w.r.t. OS-specific packages (e.g., /tools/omni_022/CGI/) just after the update so as not to break auto.pl?

Thank you!

A napp-it update via About > Update usually force a logout/login with a kill of all running napp-it processes, so this should be usually not a problem.
 
I'm adding another disk to Napp-it (passed through via ESXi) just for snapshots. I'm wondering if there is a way to tell Napp-IT to create snapshots onto my new disk?

Also, it looks like there is a way to manage snapshots of my VMs through the napp-it interface? Jobs --> ESXI Hot-snaps

Thanks for any help with this

No, not possible. A ZFS snapshot is due the CopyOnWrite filesystem simply a freeze of the current pool state, not a copy of any data.

The ESXi snapshot option is there to do an ESXi hotsnap with its hot memory state prior the ZFS snap that is like a sudden power off to include the ESXi snapshot file into the ZFS snap. This will allow an ESXi hot restore to the running/online VM state after a ZFS snap restore.
 
Last edited:
A napp-it update via About > Update usually force a logout/login with a kill of all running napp-it processes, so this should be usually not a problem.
It's a problem every time now, and I'm saying there's a repugnant lack of grace in napp-it allowing auto.pl to become broken by an update and requiring a log in to fix it.
 
auto.pl is just a cronjob that is executed example every 15 minutes to check for any jobs that are due to execute like a replication. So auto.pl is uncritical. You probably talk about running jobs. An update/ downgrade cancels jobs as the whole napp-it environment may change resulting in troubles with running jobs. Usually cancelling a job should not be a problem. You also need the logout after an update/downgrade as all running parameters, menus or other internal parameters from the old menu state may be no longer valid.

What is your OS as a logout is usually forced after an update/downgrade?
I have had only some problems with Linux and autologout.
 
Hi Gea!

I recently discovered a problem with snap jobs. As of version 17.06free 'del zero' is not working anymore. None of my snap jobs are removing empty snaps after I upgraded.

Is this a known issue?
 
It seems that the format command is hanging as this is executed by napp-it to detect connected disks. Try the format command and a zpool list at console to check behaviour. The format command can be ended after listing with ctrl-c

If format or zpool status hang, remove the new disks and insert disk by disk, wait a little and check if format is listing the disk to find a bad disk.

If format is listing all disks but stops at a special disk for a while remove it.
Update of napp-it to current is an option but would not help as this seems a disk problem.

So I know which disk is bad and I am planning to pull it tonight and replace.

Is there anything I need to do before pulling it? (it is not attached to any pool)

Is there anything I should do before replacing it? (clear anything, etc)

In the past, we have added drives a bunch at a time. My assumption now is I should add one drive, wait for it to be recognized, make sure it is good and does not lock up, then proceed to the next drive?
 
On modern hotplug capable hardware just plug/unplug.
OmniOS/OI etc will detect the change after a few seconds.

iostat keeps the removed disks in its inventory until next reboot but this does not matter.
 
I got this:

60dc21112f6074850bde37a2c2f42797.png


Does that mean the HDD is dead?

Also how to set email alerts? I tried entering my gmail adress but how can nappit need to know my email password account to send me email?
I tried to test mail & TLS (? don't work)
 
I got this:

60dc21112f6074850bde37a2c2f42797.png


Does that mean the HDD is dead?

Also how to set email alerts? I tried entering my gmail adress but how can nappit need to know my email password account to send me email?
I tried to test mail & TLS (? don't work)

Yup, bad disk, you will need to replace (you can do inside of Napp-IT, and it will resilver)

as for the email part, curious to hear from Gea about this as well. (I dont recall if it is part of the licensed/paid add-on's, but I would like this as well)
 
Last edited:
You can set a mailserver and password in About > Settings
then create an alert job that uses this settings (unencrypted port 25)

If you want to use encrypted mails example with Gmail, you must install SSL and TLS on your OS,
see https://www.napp-it.org/downloads/tls.html

Then switch mail to TLS in menu Jobs.
Your alert will then usev TLS
 
Gea,
Have you looked at SmartOS? It is made for virtualization.

Regarding Bryan Cantrill and Solaris; OpenIndiana is not Solaris. I view this as if a Linux distro is stopped developing. There are other Linux distros. Illumos is open and thriving.
 
When Oracle closed ZFS and OpenSolaris in 2010, the last Opensource ZFS and OpenSolaris bits were forked in the Illumos project and since then developped independently from Oracle as a common effort mainly by firms like Delphix, Joyent owned by Samsung (SmartOS), Nexenta or community projects like OmniOS or OpenIndiana.

Some of the distributions like OmniOS, OpenIndiana or SmartOS are OpenSource. They all share the same Illumos as base for the distribution, similar to Linux development and its distributions like Debian or CentOS but with a different focus on use cases.

While OmniOS is a minimalistic stable distribution for a just enough ZFS server, OpenIndiana adds a lot of services and the Mate GUI as an option for additional use cases. SmartOS is focused as a Cloud OS with impressive virtualisation options around KVM, Solaris zones, LX zones and Docker support. SmartOS is completely running from RAM/ readonly USB sticks with limited options in the global zone. This hinders SmartOS to be a good base for a pure storage server compared to OmniOS or OI (or would require some work to make global zone settings persistent on a datapool)
 
Quick question:
I have an incoming P3700 (400GB) to be configured as an SLOG device. Napp-it sits on top of omnios in an ESXI VM. Is there a significant performance benefit if I passthrough P3700 directly to the VM as an SLOG vs. having ESXi manage it and provision a portion for SLOG ?
 
Maybe not too significant but latency would be higher as you access the disk over the ESXi driver via a ZFS filesystem as a file on a vmfs filesystem. If you add concurrent writes from other use cases ex cache or other VMs it may become significant.

Best performance for VMs would be using the P3700 as a datastore for VMs without an additional Slog using the onpool ZIL then.
 
hoi solarish peeps
i have made my first steps with a solarish nas-box.
first try was solaris11.3 with napp-it.
pool was created, zfs smb share made and the smb user created.
(the smb user was created with the same properties as the user on the windows machine accessing the share.)
right after that i entered the smb adress on the windows machine and was presented with the share. moved some data and such.
it worked nice.

but i nulled the box and installed openindiana OI-hipster-gui-20171031 and napp-it.
again i created the pool, the zfs smb share, the smb user, just like in solaris11.3
unfortunately the box now asks for credentials when i access the share.
i have to enter the very same credentials i am logged on with.

am i tricked by my memories and the solaris11.3 box also asked for my credentials
or is there a point i missed?
 
I have not tried the new OpenIndiana but in the past this worked also for the free Solaris forks.
What you can do in Windows

- map a drive letter to the share
- save credidentials with reconnect
 
I have just tried
- Update OI 2017.04 -> 2017.10
- create a user xx
- share a filesystem

On Windows 10
- create a user xx with same pw and change user to xx
- connect to OI
I was able to access the filesystem without entering credidentials
 
no luck here.
i also tried some things
added another local user in napp-it
added the same user in win10
access is prompting for password
even created another zfs share. same result. damn...

silly me even tried to find the logfiles for that smb service.
nothing to see there. all is fine. so i do not have an error though the login is not as expected.
 
Have you modified any OS/idmap setting?
Win 10 Home or Pro (I used Pro)
 
no mapping modified. using pro editions.
i now have three local users: root, user01 and the later created user02
all except root have a SMB password
user01 and user02 are in unixgroup staff

there are the three SMB groups: administrators, powerusers, backupoperators

the SMB-UNiX mapping shows:
wingroup-powerusers@builtin ->unixgroup:staff

EDiT: while looking at my SMB User and Group-management. (Without Unix-System-user ex. root or napp-it)
i found a difference: the later created user02 has a windows SiD user01 not. user02 has to enter credentials
every time accessing the share either.
 
Last edited:
gea
i already deleted and recreated the users. even on the windows system. also with total different usernames and passwords. just in case...
no luck.
 
I have uploaded a new preview ova template for ESXi 5.5-6.5
to deploy a virtualized ZFS storage server on ESXi.

Disk size 40GB/ thin provisioned 4,5GB
e1000 (management) + vmxnet3 (data) vnic
OmniOS 151024 CE stable with NFS, SMB and FC/iSCSI
napp-it 17.06 free/Nov. edition
Open-VM-tools, midnight commander, smartmontools 6.6 etc
TLS mail enabled
Basic tuning

HowTo: http://napp-it.org/doc/downloads/napp-in-one.pdf
Download: napp-it // webbased ZFS NAS/SAN appliance for OmniOS, OpenIndiana, Solaris and Linux : Downloads
 
I have been trying to get two intel DC P3500 NVME working with ESXi and a virtualized Napp-it instance. The first one works without problems when passing through to the napp-it VM, but when I add the second as a passthrough PCIe device the first one stops working.
Have tested this with latest versions of ESXi 5.5 and 6.5. Tested on two different supermicro superserver 1028R-WTNR. Passing through the two NVME disks to a windows 2016 server VM works without problems. Using the NVME disks in ESXi directly also works without problems.

This is the error I am seeing in dmesg:
[ID 259564 kern.info] nvme0: NVMe spec version 1.0
nvme: [ID 369395 kern.warning] WARNING: nvme0: command 1/0 timeout, OPC = 6, CFS = 0
nvme: [ID 369395 kern.warning] WARNING: nvme0: command 2/0 timeout, OPC = 8, CFS = 0
nvme: [ID 988005 kern.warning] WARNING: nvme0: ABORT failed with sct = 0, sc = 0
nvme: [ID 596656 kern.warning] WARNING: nvme0: IDENTIFY failed with sct = 0, sc = 0
nvme: [ID 318795 kern.warning] WARNING: nvme0: failed to identify controller
genunix: [ID 408114 kern.info] /pci@0,0/pci15ad,7a0@16/pci8086,3705@0 (nvme0) down

Has anybody come across something like this?
 
Back
Top