OpenZFS NAS (BSD, Illumos, Linux, OSX, Solaris, Windows + Storage Spaces) with napp-it web-gui

Also, is replication suppose to work for iscsi/comstar luns? I have tried importing or creating the LU on the backup server for the replicated volume but I get stfadm errors. This is after setting the replication job to manual.
 
x-cimo said:
Ok i though express was the "free one" I guess it's the same now.

I'll give Solaris 11 a shot with napp-it :)

My hardware is on it's way.

Btw, would you consider making a less expensive or not a yearly paid fee for the extras for napp it? (Possibly for home use only? i.e: one time $150)
Click to expand...

I have not considered yet but may offer 100 Euro for
- all extensions and for 2 years per server or
- one extension and 4 years for those who needs them at home.
 
Ja'Man said:
As you can see, there is a slash after the email user before the @ sign in the "to" field. Is this normal??
Click to expand...

In perl the @ refers to a list or array. If you need the character you must mask with \@
So this is needed.
 
Ja'Man said:
Also, is replication suppose to work for iscsi/comstar luns? I have tried importing or creating the LU on the backup server for the replicated volume but I get stfadm errors. This is after setting the replication job to manual.
Click to expand...

Replication clones a dataset with all ZFS properties, snaps and volumes.
It does not clone system settings like Comstar settings.
Ex If your LU refers to a file or volume, you must set Comstar on target server manually..
 
Ex If your LU refers to a file or volume, you must set Comstar on target server manually..
Click to expand...

Can you tell me what steps then?
I can see the volume under Disks->Volumes ( I used a Vol LU target on the sender machine)
If I try to Create Vol LU and point to that volume I get "Error creating sbdadm: data file error".
If I try to import the LU I get "Error importing stmfadm: meta file error" .
Also have tried with snaps.
I have installed the iscsi and turned on Comstar. I have created the target group and the target.
What am I missing?

Thanks.
 
Ja'Man said:
Can you tell me what steps then?
I can see the volume under Disks->Volumes ( I used a Vol LU target on the sender machine)
If I try to Create Vol LU and point to that volume I get "Error creating sbdadm: data file error".
If I try to import the LU I get "Error importing stmfadm: meta file error" .
Also have tried with snaps.
I have installed the iscsi and turned on Comstar. I have created the target group and the target.
What am I missing?

Thanks.
Click to expand...

Have you disabled read only for the replicated dataset?
 
BTW, I have tested and mounted the NFS replication without issues, but the iscsi vols do not show up under zfs folder to unmark readonly, etc.
 
I'm having difficulty setting up my initial ZFS filesystems with SMB and NFS access, and am wondering if someone could walk me through a simple case.

So I created a ZFS folder called "testfs" from within napp-it. If I set the NFS property" to "on" and a SMB share, I can access it from both NFS and SMB. So that's great. But I want to make it so that I can only access the share by one user: "nick". I've already set up this user in napp-it.

In order to do so, I went to ACL on folders, deleted the ACL "everyone@", added one for user "nick" with "modify_set".

After doing so, I can access via SMB with user "nick" just fine, and other users get access denied, so that's great.

But now when I try to access via NFS with user "nick", I get access denied.

I suspect I'm missing some small critical step, but am having trouble determining what I'm doing wrong. Can someone help me out? Thanks.
 
Ja'Man said:
BTW, I have tested and mounted the NFS replication without issues, but the iscsi vols do not show up under zfs folder to unmark readonly, etc.
Click to expand...

A volume is a part of a ZFS dataset that is represented as a block device.
Therefor if the ZFS dataset is read-only, all files and volumes on it are also
read only.

You can set read-only property with menu ZFS folder on datasets.

ps
I use ESXi and always NFS. It has about the same performance like iSCSI
but if you hold multiple VM's on a NFS shared folder you can backup,
restore from snaps, clone etc all VM's on file level either locally or remote
because NFS is a file based service.

(I share all NFS folders also per SMB fore easy access, even to snaps via
previous version)
 
DarkNovaNick said:
I'm having difficulty setting up my initial ZFS filesystems with SMB and NFS access, and am wondering if someone could walk me through a simple case.

So I created a ZFS folder called "testfs" from within napp-it. If I set the NFS property" to "on" and a SMB share, I can access it from both NFS and SMB. So that's great. But I want to make it so that I can only access the share by one user: "nick". I've already set up this user in napp-it.

In order to do so, I went to ACL on folders, deleted the ACL "everyone@", added one for user "nick" with "modify_set".

After doing so, I can access via SMB with user "nick" just fine, and other users get access denied, so that's great.

But now when I try to access via NFS with user "nick", I get access denied.

I suspect I'm missing some small critical step, but am having trouble determining what I'm doing wrong. Can someone help me out? Thanks.
Click to expand...

ACL and user based access is SMB only.

With NFS at least V.3 your access is restricted on a per host base and
Unix UID's. Therfor you must either allow everyone for NFS or give
NFS full access on share level for a special host

see NFS share options like (Solaris 11 is different)
rw=host1:host2:host3,root=host1:host2:host3,ro=host4 ZFS

I always use NFS only as ESXi datastore on a per host base or in closed networks with full access
and SMB for user based restrictions and do not mix both needs.
 
Captainquark said:
About poor contacts... Today, I took out the controller and put it in another PCIe slot (4x only, though). I got more errors through that (mostly Transfer errors, but also some Hard, but no Soft), but I was able to copy data to it with about 35 MB/s (says Windows Explorer), and read data with about 60-65 MB/s. That's about double the speeds I had before, even though that PCIe slot is only 4x not 8x like the one before. So I guess I'm narrowing down the problem... Could be I had some ESD while connecting it? Dunno... Problem is more, how do I prove the vendor that the port is not working correctly?

Strange thing is still, I cannot shut down my box anymore, it always gets stuck halfway. Any idea in which log I could try to read some hints as to why this happens all the time?

Thanks,
Cap'
Click to expand...

I have decided to RMA my mainboard and the controller at the same time (different vendors), as one of them is probably broken somehow, but there's no way I could find out for sure which one. It ran much better in the x4 slot, but not perfect (had more Transfer errors and still could not reboot), so there is a chance that both are defective. This will take about 3 weeks :-( so it will take a while until I can report back. Meanwhile, thanks to all for their valuable input!
 
What is the deal with snapshot when using ZFS?

People recommand to do the following:

take esxi snapshot including memory&quiescing guest FS.
take ZFS snapshot
delete esxi snapshot.

But why? Hot snaps have always been working well on raid5, is this something specific to zfs?
 
Gday _Gea

having some weirdness with shares

Create a pool
Create a folder and share it via SMB
Nothing appears in windows

But if I click on the share and turn it off

Capture6.JPG

.
Capture7.JPG


Then back on it works
It then pops right up in windows and works a treat

Capture8.JPG


Capture9.JPG



folder was created with guest access initially
Capture10.JPG


.
 
I noticed the same thing on my NAS, always had to do it twice as you describe it. Thought it's either a bug or by design, so I didn't care too much about it... would be interested though what it is!
 
x-cimo said:
What is the deal with snapshot when using ZFS?

People recommand to do the following:

take esxi snapshot including memory&quiescing guest FS.
take ZFS snapshot
delete esxi snapshot.

But why? Hot snaps have always been working well on raid5, is this something specific to zfs?
Click to expand...

You don't want to leave snaps on esxi any longer than necessary. So, you take an esxi hot-snap, giving you a (reasonably) consistent snapshot. You then take a zfs snapshot. Once that has been done, you can delete the esxi snap, since the zfs snapshot is read only. It isn't strictly necessary to do it this way, it just improves your chances of getting a good backup.
 
Ok, figured out the problem. I had the iscsi vol sitting on the root of the pool, didnt put it under a zfs folder...
So after doing that, replication and remounting on the target server worked great!
Thanks as always for your help Gea and writing this great program! My client will be buying the repl extension by end of the day.

p.s
I agree that in all by benchmarks iscsi and NFS are pretty much neck and neck when using vmware, and of course the other advantages of windows sharing on them, etc. (thats what I use in my lab)
However, with xenserver it is different. Iscsi is usually much better, especially with a little tuning (jumbo frames, etc). Not sure if its from better iscsi implementation in their kernel or what. And because they use lvm's on their storage, you end up with one big file for all the vm's, taking away most of the advantages of sharing the back end. Again why replication is very important for me.
For those that have not heard of xenserver, its way cheaper than vmware when using a pool of servers. In fact its free with live motion and many other features, and to add most other features still way cheaper in comparison. But lack of general use makes it much harder to tshoot problems on your own. Vmware is better for sure except for the price.

Thanks again,
Jamie
 
see NFS share options like (Solaris 11 is different)
rw=host1:host2:host3,root=host1:host2:host3,ro=host4 ZFS
Click to expand...

Any plans on implementing this in your gui? Could be helpful for some...
 
Captainquark said:
I have decided to RMA my mainboard and the controller at the same time (different vendors), as one of them is probably broken somehow, but there's no way I could find out for sure which one. It ran much better in the x4 slot, but not perfect (had more Transfer errors and still could not reboot), so there is a chance that both are defective. This will take about 3 weeks :-( so it will take a while until I can report back. Meanwhile, thanks to all for their valuable input!
Click to expand...

Radical measure! If you haven't disassembled your server yet you could try cleaning your controller contacts using an pencil eraser. Gently rub them on a 90 degree angle with the contacts direction to prevent ripping them off. Then install the controller and see the results.
Be aware that this is only half a measure because the whole chain of contacts should be cleaned (motherboard slots, cables, disk contacts, etc.
Best luck!
 
hotalot said:
Radical measure! If you haven't disassembled your server yet you could try cleaning your controller contacts using an pencil eraser. Gently rub them on a 90 degree angle with the contacts direction to prevent ripping them off. Then install the controller and see the results.
Be aware that this is only half a measure because the whole chain of contacts should be cleaned (motherboard slots, cables, disk contacts, etc.
Best luck!
Click to expand...

I know, but I contacted the vendors first,described the problem and they agreed that I should send it in, they'll see what they can do. It's not an agreement that they will change it, but it's the best I could get atm.
 
Captainquark said:
I noticed the same thing on my NAS, always had to do it twice as you describe it. Thought it's either a bug or by design, so I didn't care too much about it... would be interested though what it is!
Click to expand...

Bug with Solaris 11 (smbshare not working when creating a dataset) confirmed.
Fixed in next nightly
 
danswartz said:
I figured out why it isn't working! My seagate SAS drive is NOT generating a string:

=== START OF INFORMATION SECTION ===

(no idea offhand how to fix this though...) If you want the first N lines of the output, I can certainly post that...
Click to expand...

I recently did an OpenIndiana+Napp-It installation with a bunch of SEAGATE and FUJITSU 10k SAS2 drives and ran into the same error.

_Gea said:
currently I check if a drive check returns the above string, otherwise I check for another disk type

if (!($r=~/=== START/s)) {

Easiest way is to add a OR check for another string from a successfull check like
if (!($r=~/=== START|another successfull string/s)) {

You may also post the string that is unique for a successfull check (contains sn, temp and health)
Click to expand...

Using Gea's suggestion above, I took a look at the standard output and noticed it followed the following format:

Code: 
=== START OF INFORMATION SECTION ===
Device Model:     [No Information Found]
Serial Number:    [No Information Found]
Firmware Version: [No Information Found]

However, on our SAS drives, when you run "/usr/sbin/smartctl -a -d scsi -T permissive /dev/rdsk/$disk_id", I also noticed that there is no === START line, but instead the first line seems to consistently start with "Vendor:", regardless of the brand of SAS drive I threw at it.

I'm not great with Perl, but I initially tried adding in directly a second test scenario for Vendor as Gea suggested, but since the check for "=== START" is still considered "successful" even when it's returning "[No Information Found]" this didn't really help.

I took a small liberty and re-ordered the check process, testing SCSI first instead...

Edit starting at Line 95 of get-disk-smart.pl:
Code: 
      # device must be specified!  (sat, scsi, ata),
      # try scsi first (SAS2)

           $r=`sudo /usr/sbin/smartctl -a -d scsi -T permissive /dev/rdsk/$disk`;          # default = sat= sat,16 is needed for 48bit ATA support
                                                                                             # default (16) is reported to have problems on some intel ahci
           # print "<hr>smartctl -a -d scsi -T permissive /dev/rdsk/$disk <hr> $r\n";         # debug, needs html header line 32
           # print "--------------- scsi? -> $r\n";

           $type="scsi"; 
           $k="$id\_smart_dtype";
           $disk{$k}="$type";

           $k="$id\_smart_type";
           $disk{$k}.=" $type";                                                           # add type

           # opt read temp extra/directly
           if (!($r=~/Temperature/s)) {
              $r.= `sudo /usr/sbin/smartctl -l scttempsts -d scsi -T permissive /dev/rdsk/$disk`; #sat
           }


      ## if no START or Vendor is found, try SAT

      if (!($r=~/=== START|Vendor:/s)) {                                                           # === START OF INFORMATION SECTION ===

           $r=`sudo /usr/sbin/smartctl -a -d sat,12 -T permissive /dev/rdsk/$disk`;

           # print "<hr>smartctl -a -d sat,12 -T permissive /dev/rdsk/$disk <hr> $r\n";

           $type="sat,12";
           $k="$id\_smart_dtype";
           $disk{$k}="$type";
          $k="$id\_smart_type";
           $disk{$k}.=" $type";

           # opt read temp extra/directly
           if (!($r=~/Temperature/s)) {
              $r.= `sudo /usr/sbin/smartctl -l scttempsts -d sat -T permissive /dev/rdsk/$disk`;
           }
      }

This should check if the disk is SCSI (SAS) first, if it sees either === START or Vendor, then it worked and continues. If not, THEN it will go back and check for a SAT,12 device.

This seemed to work for me. I now am able to see smart_health, smart_sn and smart_temp on my SAS drives. The only thing I haven't gotten working is showing smart_model yet.

Anyway, if you're still using your SAS drives and want SMART info viewable via napp-it, this code update should hopefully work for you as well.
 
Hello dsumike

thank you for your testings.
I will try it with my disks and controllers and add your suggestions to next nightly
 
Can someone help me out...

I've managed to get my OI box up and running and have created (2) datastores (one for VM's and the other for "MEDIA")... I'm trying to configure SAB to use the MEDIA datastore to write to it but to be honest I don't know how...

Sorry, I'm a complete Solaris/Linux noob... when I have the file browser open, and I click on "File System" I can see both datastores and they both read the correct size.

When I configured SAB I pointed to -

Temporary Download Folder: DataShare (this is what i named it)

and Completed Download Folder" DataShare/Media

however by doing this, it created these folders in my home/username (jaysen) directory so SAB is telling me I am running out of space.
 
Anyone has an update about support for vmware tool in Solaris 11?, maybe in with the ESXi5 Update 1?

If not, which ZFS OS support vmware tool and vmxnet3 10G interface properly?

Thanks!
 
I got vmware tools working on solaris 11. Used a trick though, install solaris express 11, install vmware tools from esxi 5 U1 then perform upgrade from se 5.11 to solaris 11.11
vmware tools continue to work on solaris 11 :D
 
axan said:
I got vmware tools working on solaris 11. Used a trick though, install solaris express 11, install vmware tools from esxi 5 U1 then perform upgrade from se 5.11 to solaris 11.11
vmware tools continue to work on solaris 11 :D
Click to expand...


This is great! Are you using the vmxnet3 nics?

Do you have an idea where I can find a copy of Solaris Express 11, it dosen't seem to be available anymore...

Finally how easy is the upgrade from Express to Solaris11?
 
I wonder then if the E1000 "Legacy" ESXi nic is rate limited at 1Gbps or if it does goes over.

Let use know how the vmxnet3 is going :)

Thanks for you help!
 
basically it constantly spits out
Code: 
vmxnet3s: [ID 654879 kern.notice] vmxnet3s:0: getcapab(0x200000) -> no
everything seems to work fine so probably ok to ignore. Network speeds seem fine, but I need to do some 10Gbe tests this weekend
 
I've got a problem which is frustrating the hell out of me.
Just installed OI and Napp-It on ESXI.

Created my pool (raidz over 5 disks). Create my shares and shared via smb/nfs with nbmand turned on. Guest access is on, nothing complicated setup.

I can access the smb shares on my laptop - but not on another computer. I can connect via Windows 7 NFS on my laptop to the pool - but can't see any of the shares. However, I share a zfs folder back to ESXI for the VM's, and that works fine.

I'm really at my wits end here and need some help - I feel I'm missing something really basic, as this is NOT a complicated setup at all....
 
_Gea said:
Hello dsumike

thank you for your testings.
I will try it with my disks and controllers and add your suggestions to next nightly
Click to expand...

No problem.

As a follow-up, I was very happy having the smart information, but I noticed that it appeared entire sets of like drive types were reporting the same temperature, which looked really high (65 C - 70 C).

Further investigation showed that the script was actually picking up the drives trip temperature instead of the actual current temperature. See print out below:

Code: 
SMART Health Status: OK

Current Drive Temperature:     23 C
Drive Trip Temperature:        68 C
Elements in grown defect list: 0

I made an additional adjustment in the get-disk-smart.pl script on approximately Line 234 (after my previous modifications) or Line 232 prior to modifications:

Code: 
           #temp
           if ($s=~/Temperature/i) {

              if ($s=~/Max Temperature/i)   { next; }              # no min/max
              if ($s=~/Temperature Limit/i) { next; }              # no limit count
              [COLOR="Red"]if ($s=~/Trip Temperature/i) { next; }               # no Drive Trip Temperature (dsumike modification)[/COLOR]

                $s=~s/.*  +//;
                $s=~s/ .*//;                                       # only temp not temp + min/max

                $k="smart_temp";
                if ($s=~/\d$/) { $s.=" �C" }

                $sv{$k}=$s;
                next;
           }

Once I added this additional check to "skip" the Drive Trip Temperature when parsing, the page now displays the current temperature correctly for my SAS drives.
 
hello dsumike

It is hard to care about all the differences with Smartmontools and different disks and controllers
as well as the differences between Illumian, OI and Solaris 11 (and eventually SmartOS.)

It is a good sign that more and more users look at the sources of napp-it and help to fix bugs.
I have done my best to keep the Perl code as simple as possible and to keep it user editable so
that you do not need more than basic scripting knowledge.

If you are new to Perl: You can do horrible things with Perl but you must not.
It is a language similar to PHP and I like it especially for system management. .
see Perl in about two hours at http://qntm.org/files/perl/perl.html

Some documentation needs to be done but thats hard when sources are not yet ready.
To all napp-it users: look at the sources, ask if you have questions about dependencies.

Thanks again.
 
About ZFS >V.28

Currently, ZFS encryption is Oracle Solaris 11 and ZFS V.31+ only, so if you need, you have no options. But you loose compatibility forever with all other ZFS implementations like FreeBSD, Illumos based ones (Illumian, OpenIndiana, SmartOS), Linux or OSX which are on ZFS V.28

I do not expect newer Versions than V28 that are Oracle compatible.
Its more likely that we see features flags in future, not Versions > 28 from others than Oracle.
http://blog.delphix.com/csiden/files/2012/01/ZFS_Feature_Flags.pdf
 
_Gea said:
hello dsumike
It is hard to care about all the differences with Smartmontools and different disks and controllers as well as the differences between Illumian, OI and Solaris 11 (and eventually SmartOS.)
Click to expand...

Yeah, I completely understand, which is why I have been trying to post my small modifications. Hopefully the little tweaks make it easier for you. :)

So far I am loving Napp-It. Your hard work is very much appreciated.

Thank you!
 
so, I built 2 new domain controllers and demoted the old one and now one of my napp-its won't join.

I deleteded it fomr AD.

Maybe I can't type the right things into napp-it AD menu. Any guesses. All ADs now are 2008 R2. One thing I notice is it says. "No server suitable for synchronization found"
I put the IP for the DC that has all the fsmo roles.
 
Gea im having some trouble getting the comstar iscsi to present a volume to my clients. Im able to connect to the LUN fine, I have it set as a favorite target, but in the windows iscsi initiator in the volumes and devices tab I cant get a volume to show up, I tried pressing auto configure and adding the mount point manually. Is there something common that im missing?

edit: Got it to work by adding a target view
 
Last edited:
Hi Gea, I am using Napp-it on Illumian, running a all-in-one setup on ESXi 5.0.
The dd benchmark results seemed ok but the SMB tests seemed a little slow.

All-in-one Hardware specs:
i7-2600
Asrock Z68 Extreme4 Gen3
32GB GSkill memory
LSI SAS 9211-8i
3 x 2TB WD Green (WD20EARX)
2 x 128GB Crucial m4 SSD
1 x 64GB Crucial m4 SSD
Intel Gigabit CT 10/100/1000 PCI-E Ethernet Network Adapter

The Illumian VM is allocated 8GB of memory.

Raid Type & dd Benchmark results

Raid-z : 3 x 2TB WD Green
10.24 GB in 76.5s = 133.86 MB/s Write
10.24 GB in 48.4s = 211.57 MB/s Read

Stripped : 2 x 128GB Crucial m4 SSD
10.24 GB in 26.8s = 382.09 MB/s Write
10.24 GB in 11.9s = 860.50 MB/s Read

SMB test is conducted by copying a 3GB ISO file between my desktop running Windows 7 x64 to the Illumian shares. Desktop has a 128 Corsair SSD and a 500GB WD Black HDD.

Desktop SSD copy to Raid-z share: sustained 100MBps
Raid-z copy to Desktop SSD: sustained 30MBps

Desktop HDD copy to Raid-z share: sustained 68MBps
Raid-z copy to Desktop HDD: sustained 27MBps
--------------------------------------------------------------------------------------
Desktop SSD copy to Stripped share: sustained 100MBps
Stripped copy to Desktop SSD: sustained 30MBps

Desktop HDD copy to Stripped share: sustained 69MBps
Stripped share to Desktop HDD: sustained 29MBps


I am not sure what is causing the slowness when copying from the shares to desktop, 30MBps seems slow to me especially from a stripped 2 x SSD to my desktop SSD. :confused:

I have not tested with iSCSI yet, getting a "sudo itadm not found" when I tried to Create a Target.
 
tladuke said:
so, I built 2 new domain controllers and demoted the old one and now one of my napp-its won't join.

I deleteded it fomr AD.

Maybe I can't type the right things into napp-it AD menu. Any guesses. All ADs now are 2008 R2. One thing I notice is it says. "No server suitable for synchronization found"
I put the IP for the DC that has all the fsmo roles.
Click to expand...

Have you tried different lmauth settings
use 3 with S11 and OI 151a1, otherwise 2 is suggested
 
You must log in or register to reply here.
Back
Top