OpenSolaris derived ZFS NAS/ SAN (OmniOS, OpenIndiana, Solaris and napp-it)

_Gea did you see my thread in the Virtualization forum. When i set a static IP in openindiana the DNS does not work and when i restart the machine the DNS entry i add to the resolv.conf file is empty?
 
no have not seen

after setting a manual ip you must set a dns entry
either by editing this file or via napp-it system-network-dns


add for example (free Google DNS):
nameserver 8.8.8.8

ps
Network settings of all Solaris is absolute user unfriendly and not intuitive
the author of this network setting dialog must have been on drugs when he wrote the code

but maybee i'm just not the advanced user with advanced needs for the options it can deliver

google for "solaris set manal ip" and you have 6.300.000 hits
maybee we should start a petition to OpenIndiana for usability (i doubt Oracle would care)
 
Last edited:
no have not seen

after setting a manual ip you must set a dns entry
either by editing this fie or via napp-it system-network-dns


add for example (free Google DNS):
nameserver 8.8.8.8

ps
Network settings of all Solaris is absolute user unfriendly and not intuitive
the author of this network setting dialog must be on drugs when he wrote the code

I do this and DNS does not work. Then when i restart the machine the manual nameserver entry is gone and the resolv.conf file is empty with no text. I found something online that there is a bug where if you use a static entry it still tried to get DNS from DHCP but will make the resolv.conf blank.
 
_Gea, how did you figure that one out? idmap is still somewhat mysterious for me.

BTW will this help with NFS shares, using third party NFS clients (ie. OpenText NFS Solo)? I've been struggling with my NFS Solo client evaluation that would freeze my windows 7 box whenever I try to connect to OpenIndiana's NFS server. I still need to read up on DTrace to figure out if that can help me verify if it is ACL issues.
 
I do this and DNS does not work. Then when i restart the machine the manual nameserver entry is gone and the resolv.conf file is empty with no text. I found something online that there is a bug where if you use a static entry it still tried to get DNS from DHCP but will make the resolv.conf blank.

i can understand and i have the same problems
either its buggy or with worse usability.

for me, I wait for oi stable and see how i can live with it
(can handle with napp-it)

to be clear: i do not believe i can decide about usability
- no developer can- the common user is the only person who can

but i get pain if you botch up things like IP settings on Solaris
 
Last edited:
_Gea, how did you figure that one out? idmap is still somewhat mysterious for me.

BTW will this help with NFS shares, using third party NFS clients (ie. OpenText NFS Solo)? I've been struggling with my NFS Solo client evaluation that would freeze my windows 7 box whenever I try to connect to OpenIndiana's NFS server. I still need to read up on DTrace to figure out if that can help me verify if it is ACL issues.

Unix use user id's and group id's
Windows use security id's in another format for both

if you want ti build a Unix server that can replace a Windows server you must
either have a os/filesystem that can handle security id's like NTFS and manage user and groups like Windows (would be a great option)
or you must map Windows security id's to unix user or groups (with much less capabilities)

That the way, Solaris works with Kernel based cifs and as I know, its the best available option in the Unix/ Linux world
(not perfect but there is nothing better)

Solaris can create a automatically mapping Windows user/group on they fly (on use) with a security ID -> Unix user-id/group ID
Beside this automatically mapping, you can create manual mappings via the idmap command for example to
create a mapping from domain-admin -> unix root.

I do not suppose it help with NFS if you are on NFS3 (not aware of ACL) NFS4 is ACL aware but i do not know, how and if its interact with idmap
I'm not experienced enough with NFS
 
Last edited:
Did i do it right now, i created a mirror pool with 2 drives then Add Vdev and selected the other two drives as mirror. This is what i got:

pool: rpool
state: ONLINE
scan: none requested
config:

NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
c2t0d0s0 ONLINE 0 0 0

errors: No known data errors

pool: zfs1
state: ONLINE
scan: none requested
config:

NAME STATE READ WRITE CKSUM
zfs1 ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c3t0d0 ONLINE 0 0 0
c3t1d0 ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
c3t2d0 ONLINE 0 0 0
c3t3d0 ONLINE 0 0 0

errors: No known data errors

CP81q.jpg


Here are my raidz1 results:

Is it normal for raidz to be faster than the mirror?

Km1fS.jpg
 
yes, you have a raid-10
but with a raid-10 not the sequental values are better but the IO/s
these values are more important for some use cases
 
Raid-10 of 4 drives stripes across two vdevs. Speed should be ~2x speed of one drive.

Raidz of 4 drives stripes across 3 drives. Speed should be ~3x speed of one drive.

Your benchmark results look about right.
 
PigLover, I don't think this is right. Reading from a mirror with zfs, large sequential reads should be 2X a single drive, so reading from a 4-drive raid10 gives about 4X a single drive. This is what I see in the real world, and everything I've found via google corroborates it.
 
Example from my 3x2 raid10 array of sata drives:

write: 10737418240 bytes (11 GB) copied, 48.6611 s, 221 MB/s
read: 10737418240 bytes (11 GB) copied, 25.5804 s, 420 MB/s

note that the read speed is about 2X write...
 
Last edited:
PigLover, I don't think this is right. Reading from a mirror with zfs, large sequential reads should be 2X a single drive, so reading from a 4-drive raid10 gives about 4X a single drive. This is what I see in the real world, and everything I've found via google corroborates it.

Completely different from my experience and testing - both with ZFS and hardware raids. Possible in theory, yes, but not how its actually done. You only get the read speed advantage from the striped part, no speed gain from the mirror.
 
Dunno what to tell you. Google for raidz vs raid10 and everyone says the same thing. From oracle themselves:

http://blogs.oracle.com/roch/entry/when_to_and_not_to

quote: "With 2-way mirroring, each FS block output must be sent to 2 devices.
Half of the available IOPS are thus lost to mirroring. However, for
Inputs each side of a mirror can service read calls independently from
one another since each side holds the full information. Given a
proper software implementation that balances the inputs between sides
of a mirror, the FS blocks delivered by a mirrored group is actually
no less than what a simple non-protected RAID-0 stripe would give."

http://blogs.oracle.com/relling/entry/zfs_raid_recommendations_space_performance

tends to say the same thing. in any event, i wasn't talking about hardware raids, or linux mdadm (i've been told it doesn't do read balancing, but freebsd's gmirror does.) all that aside, i'm puzzled you and amd don't see this behavior.
 
Hey guys,

just got about half way through this thread and decided to give napp-it a try. However, having just installed OI Server I'm getting the following error:

root@zfs-server:/tmp/vmware-tools-distrib# wget -O www.napp-it.org/nappit | perl
Backticks found where operator expected at - line 4, at end of line
(Missing semicolon on previous line?)
syntax error at - line 2, near "Usage:"
Can't find string terminator "`" anywhere before EOF at - line 4.
root@zfs-server:/tmp/vmware-tools-distrib#
 
Hey guys,

just got about half way through this thread and decided to give napp-it a try. However, having just installed OI Server I'm getting the following error:

root@zfs-server:/tmp/vmware-tools-distrib# wget -O www.napp-it.org/nappit | perl
Backticks found where operator expected at - line 4, at end of line
(Missing semicolon on previous line?)
syntax error at - line 2, near "Usage:"
Can't find string terminator "`" anywhere before EOF at - line 4.
root@zfs-server:/tmp/vmware-tools-distrib#

Code:
[COLOR=Wheat]wget -O - www.napp-it.org/nappit | perl
[/COLOR]

 
Hey guys,

just got about half way through this thread and decided to give napp-it a try. However, having just installed OI Server I'm getting the following error:

root@zfs-server:/tmp/vmware-tools-distrib# wget -O www.napp-it.org/nappit | perl
Backticks found where operator expected at - line 4, at end of line
(Missing semicolon on previous line?)
syntax error at - line 2, near "Usage:"
Can't find string terminator "`" anywhere before EOF at - line 4.
root@zfs-server:/tmp/vmware-tools-distrib#

you missed a minus after the O
wget -O - www.napp-it.org/nappit | perl
 
Hmm, now I feel stupid, I remember reading that exact same reply earlier in the thread. Thanks.
 
hey

Im having a problem setting up web services, i used the command thats given in the PDF and it does install with out any errors and when i log in to napp-it it says "apache-server: maintenance". so i go over to the Apache page to configure it and when i hit apache.config it gives me an error saying it cant find:
Code:
/etc/apache2/apache2.conf
so i go in to OI and i find the config under /etc/apache2/2.2/
same for php.in
I dont know how i would change this.
i have never setup Apache and php before other then for testing in windows.
What im trying to do is setup phpvirtualbox so i wont have to vnc in to use virtualbox.

thanks for the help.
 
you can edit /etc/apache2/2.2/httpd.conf with nano from the terminal

sudo nano /etc/apache2/2.2/httpd.conf

I just use the CLI so I'm not sure what the default editor is in OI, gedit?

good luck
 
ok so im having a strange problem...
i was unable to connect to the apache server so i thought a restart might be in order...well after restarting the boot takes a long time (i just thought it was because of the pool, hadnt restarted sense creating it) and then i get a black screen with the mouse courser of a stop watch and it never changes, it wont load the longin screen, BUT i can still get to napp-it over the network, SSH in to the box, and connect to the shares. i just cant get to the GUI when im at the box, but i can use X through SSH to call programs such as firefox and such.

No idea why this would be...any guess??

Thanks for all the help.
 
I have a quick question about NFS or SMB shares. I want to set up shares for my media and I would like to create three separate SMB (or NFS) shares one for TV, movies and music.

I've tried using sharemgr but the parent folder which contains the TV, movies and music folders is already shared and it wont let me share the subfolder (with NFS, couldn't figure out SMB). Is there a way to share sub-folders or should I just make a new folder and put symlinks in it and share those.

Thanks!
 
I am a little confused about some of the ACL extension features.

What is the difference between ZFS/dataset ACL and SMB share ACL?
Also, setting the ACL for individual users works great. How do I do it for groups? I don't see any option to add groups (only add local user). Does it apply to the UNIX groups? (I'm talking about the group@ ACL trivial etting). Thanks a lot :)
 
I also have (another) question about ACLs...

If the parent folder allows the "delete" permission but a sub-folder does not, why doesn't the parent permission override the sub-folder permission?

Example:

/tank/downloads (@everybody has delete access)
/tank/downloads/newFolder (@everybody does NOT have delete access)

Even though the parent folder allows me delete access, I can't delete anything from the sub-folder.

Why doesn't my permission from the parent folder override the subfolder?

Here are the listing from ls -v:
Code:
drwxrwxrwx+247 root     root         251 Aug 14 23:18 downloads
     0:user:root:list_directory/read_data/add_file/write_data
         /add_subdirectory/append_data/read_xattr/write_xattr/execute
         /delete_child/read_attributes/write_attributes/delete/read_acl
         /write_acl/write_owner/synchronize:file_inherit/dir_inherit
         /inherited:allow
     1:everyone@:list_directory/read_data/add_file/write_data
         /add_subdirectory/append_data/read_xattr/write_xattr/execute
         /delete_child/read_attributes/write_attributes/delete/read_acl
         /synchronize:file_inherit/dir_inherit/inherited:allow
Code:
drwxrwxrwx   2 1001     1001           4 Aug 10 07:51 newFolder
     0:owner@:list_directory/read_data/add_file/write_data/add_subdirectory
         /append_data/read_xattr/write_xattr/execute/read_attributes
         /write_attributes/read_acl/write_acl/write_owner/synchronize:allow
     1:group@:list_directory/read_data/add_file/write_data/add_subdirectory
         /append_data/read_xattr/execute/read_attributes/read_acl
         /synchronize:allow
     2:everyone@:list_directory/read_data/add_file/write_data
         /add_subdirectory/append_data/read_xattr/execute/read_attributes
         /read_acl/synchronize:allow
 
I also have (another) question about ACLs...

If the parent folder allows the "delete" permission but a sub-folder does not, why doesn't the parent permission override the sub-folder permission?

you must set ACL recursively
chmod -R ...
 
I am a little confused about some of the ACL extension features.

What is the difference between ZFS/dataset ACL and SMB share ACL?
Also, setting the ACL for individual users works great. How do I do it for groups? I don't see any option to add groups (only add local user). Does it apply to the UNIX groups? (I'm talking about the group@ ACL trivial etting). Thanks a lot :)


- share acl, see http://www.windowsecurity.com/articles/Share-Permissions.html
- SMB groups are not unix groups, They are created/managed via menü user - smb groups
- not all options are ready in ACL extension (under development, have less time at the moment)
 
I have a quick question about NFS or SMB shares. I want to set up shares for my media and I would like to create three separate SMB (or NFS) shares one for TV, movies and music.

I've tried using sharemgr but the parent folder which contains the TV, movies and music folders is already shared and it wont let me share the subfolder (with NFS, couldn't figure out SMB). Is there a way to share sub-folders or should I just make a new folder and put symlinks in it and share those.

Thanks!

with Kernel CIFS or NFS server, sharing is a property of a ZFS filesystem
-> nested shares are not possible -> create a ZFS for each share

you may use SAMBA where you can nest shares, but its not as fast and easy and
not as Windows like with ACL's than CIFS
 
Thanks for the help Gea.

I've decided to switch over to CIFS (SMB) for everything instead of mixing NFS and CIFS. The ACLs just end up getting messed up since a lot of applications don't work well with the Solaris NFS v4 ACLs. (ie. they end up doing [chmod 755] which destroys the ACLs that I had previously created, etc)
 
just read the following news:
Integrating KVM into the Illumos Kernel (base of OpenIndiana and next Nexenta)

from http://smartos.org/
SmartOS: The Complete Modern Operating System:

SmartOS incorporates the four most revolutionary OS technologies of the past decade — Zones, ZFS, DTrace and KVM — into a single operating system, providing an arbitrarily observable, highly multi-tenant environment built on a reliable, enterprise-grade storage stack. With the introduction of KVM in SmartOS, you no longer have to give up the power of an enterprise-grade operating system in order to run legacy applications and stacks.

SmartOS turns any server into a highly efficient hosting platform for multi-tenant, machine-to-machine, or storage applications. SmartOS offers unique, innovative tools for application developers, service providers and data center operators – tuned and hardened for modern datacenter deployment....

Sounds very interesting.

more
http://www.joyent.com/products/smartos/smartos-faq/
http://dtrace.org/blogs/bmc/2011/08/15/kvm-on-illumos/
http://www.readwriteweb.com/enterprise/2011/08/joyent-brings-kvm-to-smartos-f.php
 
Last edited:
just read the following news:
Integrating KVM into the Illumos Kernel (base of OpenIndiana and next Nexenta)

from http://smartos.org/
SmartOS: The Complete Modern Operating System:

SmartOS incorporates the four most revolutionary OS technologies of the past decade — Zones, ZFS, DTrace and KVM — into a single operating system, providing an arbitrarily observable, highly multi-tenant environment built on a reliable, enterprise-grade storage stack. With the introduction of KVM in SmartOS, you no longer have to give up the power of an enterprise-grade operating system in order to run legacy applications and stacks.

SmartOS turns any server into a highly efficient hosting platform for multi-tenant, machine-to-machine, or storage applications. SmartOS offers unique, innovative tools for application developers, service providers and data center operators – tuned and hardened for modern datacenter deployment....

Sounds very interesting.

more
http://www.joyent.com/products/smartos/smartos-faq/
http://dtrace.org/blogs/bmc/2011/08/15/kvm-on-illumos/
http://www.readwriteweb.com/enterprise/2011/08/joyent-brings-kvm-to-smartos-f.php

That looks VERY interesting. Downloaded the ISO and plan in trying it out on our new storage server.
 
Thanks for information about kernel CIFS/NFS sharing. I was hoping to avoid using a zfs folder since it appears to take time to move files from the zfs folder to a sub-ZFS folder. I was wondering why moves are not instant between ZFS folders. I guess it is actually moving the information on the drive?
 
Yes, think of the datasets as linux filesystems - you can't just move a directory entry or link or whatever...
 
Would a generic 2 port sata controller work with esxi passthrough? I need to add my cache drive to my pool and would like to do it cheaply.
 
Easier said than done! I have to buy one and would like to buy one that works and works with openindiana ;)

well, didn't know that from your previous post. STF, there are several relatively cheap (under $100) HBA that can be found on ebay that work just fine passed thru from esxi with OI as the guest.
 
Does anyone here know why Open Indiana VMs cannot boot with more than 2 virtual cpus? Is this a bug that will be fixed soon, or some inherent limitation?
 
Back
Top