OpenSolaris derived ZFS NAS/ SAN (OmniOS, OpenIndiana, Solaris and napp-it)

Hey Gea, is it possible to nest ZFS folders with different pools?

Since our MD3000i had a RAID controller die we are looking at building an IO/Napp-It bare metal server and we will have a mix of disks. We currently have:

15 - 420GB SAS 15k drives
8 - 1TB nearline SAS 7200 drives
7 - 2TB SATA drives

We also want to test out some SSD storage for our database shortly, so I would like to have something like this under NFS:

/pool/DB (mirrored pool 15k SAS drives)
/pool/VM (raidz-3 pool of the nearline SAS drives)
/pool/backup (raidz-2 pool of the SATA drives)

It wouldn't kill me to have to create a /poolDB, /poolVM, and /poolBackup, but I don't really want to have to manage each new connection or pool in each VMWare server we have.

If I get some free time today I'll try something in virtualbox, but I thought this might be a good place to get a yes/no answer before I spend what little spare time I have testing this out.
 
Yo,

Just distroyed a pool called Qmedia1 as I wanted to use different disks but when I try to recreate it I get error that Qmedia1 allready exists....never had this problem before!
And I need it to have the same name as before for my XBMC to find all my movies!
 
brianmat said:
Hey Gea, is it possible to nest ZFS folders with different pools?

Since our MD3000i had a RAID controller die we are looking at building an IO/Napp-It bare metal server and we will have a mix of disks. We currently have:

15 - 420GB SAS 15k drives
8 - 1TB nearline SAS 7200 drives
7 - 2TB SATA drives

We also want to test out some SSD storage for our database shortly, so I would like to have something like this under NFS:

/pool/DB (mirrored pool 15k SAS drives)
/pool/VM (raidz-3 pool of the nearline SAS drives)
/pool/backup (raidz-2 pool of the SATA drives)

It wouldn't kill me to have to create a /poolDB, /poolVM, and /poolBackup, but I don't really want to have to manage each new connection or pool in each VMWare server we have.

If I get some free time today I'll try something in virtualbox, but I thought this might be a good place to get a yes/no answer before I spend what little spare time I have testing this out.
Click to expand...
dont ... just don't. you can mix drives and raid types in a pool but not within datasets, least not that i'm aware of. only mix drive types and or raid levels if you absolutely must though.
 
siamsquare said:
Yo,

Just distroyed a pool called Qmedia1 as I wanted to use different disks but when I try to recreate it I get error that Qmedia1 allready exists....never had this problem before!
And I need it to have the same name as before for my XBMC to find all my movies!
Click to expand...
the Qmedia1 that you destroyed, are you positive you don't need anything on it?

if the answer to that is no, you hate that data and want it gone forever then run the zpool create command again with the -f flag.
 
madrebel said:
the Qmedia1 that you destroyed, are you positive you don't need anything on it?

if the answer to that is no, you hate that data and want it gone forever then run the zpool create command again with the -f flag.
Click to expand...

Found another sollution....created a pool called Qmedia3 and destroyed that! Imported the Qmedia3 as Qmedia1 and it worked!
Busy coppying 16TB.....
 
Hi Gea,
Please, please for the love of the greater power you believe in - check each and every password input and variable you handle within your gui.
I wasted another 30 minutes rummaging around with my two appliances, before I realized that there might me an error caused by my password length and or special characters.
The bug I'd like to report concernes the replication feature, when adding an appliance to the group.
I think it would be useful if you could make sure that passwords consisting of 32 characters or more (alphanumeric and special characters) are handled correctly.
Apart from that, I'm still a very satisfied customer :)
Have a nice evening,
-Frozen
 
Frozen.Shots said:
Hi Gea,
Please, please for the love of the greater power you believe in - check each and every password input and variable you handle within your gui.
I wasted another 30 minutes rummaging around with my two appliances, before I realized that there might me an error caused by my password length and or special characters.
The bug I'd like to report concernes the replication feature, when adding an appliance to the group.
I think it would be useful if you could make sure that passwords consisting of 32 characters or more (alphanumeric and special characters) are handled correctly.
Apart from that, I'm still a very satisfied customer :)
Have a nice evening,
-Frozen
Click to expand...

Such long passwords with any special characters may be a problem.
napp-it itself use up to 16 characters for the management users admin and operator with some special characters.

The values for the underlying Unix users are not controlled by napp-it but depends on versions and services. If you use NexentaCore, your default is max 8 characters. On newer versions it is setable but i would not expect that more than 12-16 char are supported. For cross platform needs (Mac, Windows, Unix, Web) you must avoid non 7bit ASCII characters as well.

http://superuser.com/questions/148971/what-is-the-max-length-of-password-on-unix-linux-system
http://blog.mc-thias.org/?title=solaris-10-password-length-limitation&more=1&c=1&tb=1&pb=1
http://www.itworld.com/endpoint-security/277046/how-enforce-password-complexity-solaris


Not to forget:
napp-it is intended for secure networks.
For example: Replication is speed optimized and not encrypted. Web-access is also not encrypted. Why do you need a pw-length of 32 char then?
 
Last edited:
Congratulations, you outmaneuvered me. I hope (native) SSL support for both mail and web-access are on your roadmap somewhere after Version 1.0. Otherwise you should make a more prominent remark on your website about napp-it's intended use.
Education or Development may include insecure networks. Not anyone is willing or has the budget to isolate their infrastructure to compensate for this.
-Frozen
 
@Frozen.Shots

Wow. It's not expensive or difficult to isolate your infrastructure. Just learn some commonsense and how about you actually put in some effort rather than expecting things to be handed to you.
 
Frozen.Shots said:
Congratulations, you outmaneuvered me. I hope (native) SSL support for both mail and web-access are on your roadmap somewhere after Version 1.0. Otherwise you should make a more prominent remark on your website about napp-it's intended use.
Education or Development may include insecure networks. Not anyone is willing or has the budget to isolate their infrastructure to compensate for this.
-Frozen
Click to expand...

I always separate my networks via vlan in a separate management network, a SAN network to connect my storage and backup machines and a more or less unsecure lan for filers - optionally with additional firewall settings This is the most important security rule.

SSL and https are already possible if you install the perl modules and minihttpd certificates manually. But do not expect any real security advance from https with private certificates in a switched network.

So if you look at security:
- use vlans, manage only locally or via isolated management networks, look at who is allowed to access the network,
optionally disable napp-it and ssh when not needed
- replicate only via vpn or via isolated storage networks
- do not use your regular email account for error mails (napp-it must store and send your pw in cleartext), use a forwarder

- Restrict access physically. Any ip attack needs skill, you can order a thief more easily.

But if you need to have your appliances including management and replication in the internet - you should not use napp-it
or other remote management tools.
 
Frozen.Shots said:
Congratulations, you outmaneuvered me. I hope (native) SSL support for both mail and web-access are on your roadmap somewhere after Version 1.0. Otherwise you should make a more prominent remark on your website about napp-it's intended use.
Education or Development may include insecure networks. Not anyone is willing or has the budget to isolate their infrastructure to compensate for this.
-Frozen
Click to expand...

That's a pretty harsh way to respond to a guy who has built a wonderful framework for managing a ZFS-based NAS host and put it out there absolutely free for the community - and then sticks around and offers free help/support/advice to people who use it.

Tell you what - all the scripts are all there. Instead of griping at him about things you want added, why not help make the changes you want and send them back to Gea?
 
There's a fine line here. Don't just jump on the guy, because what he's asking isn't an unreasonable set of features to want. Being able to have the server configured with more security is certainly desirable.

From an admin's point of view it can be a daunting process trying to determine whether or not it can be made more secure. Asking the question in ways that don't antagonize the existing users and developers is certainly something to try to avoid. But mistakes get made sometimes.

I'm sure from a developer's standpoint it's even messier, and the cost of implementing it has to be weighed against other demands. Sometimes it's just 'too hard' to implement all of the end-case demands. Security can be a real hassle to get configured properly.

But in the meanwhile it certainly can't hurt to address the topic. Perhaps patches can be created and submitted, as a way to help the project overall. But yelling at the guy for how he's asking is probably just as unhelpful as the way the request has been perceived.
 
Yo,

I'm trying to give the AIO a shot and using the tutorial....I have a 60GB SSD as boot and installed ESXi 5.1 on it! Can I use the same SSD to install OI or do I need a second one?
 
siamsquare said:
Yo,

I'm trying to give the AIO a shot and using the tutorial....I have a 60GB SSD as boot and installed ESXi 5.1 on it! Can I use the same SSD to install OI or do I need a second one?
Click to expand...

Yes you can and I do it this way also.
 
siamsquare said:
So I can slam Unraid,OI + Napp-it and maybe Win7 all on one SSD?
A pitty I can't mirror the SSD without a raid card just to be safe!
Click to expand...

You can use a driverless Sata Raid-1 enclosure like
a Fantec MR-25D or Raidsonic SR2760-2S-S2
 
When I need napp-it remotely[from my home],I just connect to my corporate Windows machine via RAC [Remote Admin Control]. The said win PC and the whole network sits behind BSD based router/fw.
 
Thanks for the help Gea,

How does one get out of vm's terminal screen without hitting ctrl+alt+del in esxi 5.1?
 
Yo,

Need some help !
Busy installing the AIO and hit a wall!
I installed OI as VM on ESXi 5.1 and installed VMware on OI and also Napp-it! OI gets an IP adress from DHCP router but when I try to connect to the Napp-It terminal (192.168.1.6:81) it's not found!
I also see that ESXi only finds 1 network adapter on my SM X9SCM-F, I know there are 2 different ones...still no drivers for the other one?

thanks
 
Last edited:
In the all in one setup is transfer speed between ESX and napp-it NFS share limited by the speed of physical NIC?

I have ESX connected to the NFS share on Solaris VM, but the transfer speed is flat ~930 Mbps. Is there any way to work around this?
 
levak said:
If you use vSwitch then the physical NIC should not be a limit...
Click to expand...

I have a dedicated vSwitch for ESX to Solaris traffic. The only two things connected to it are VMkernel port and Solaris VM which uses VMXNET3 adapter.
 
Little off-topic but..Few days ago ,friend of mine take the opportunity to install napp-it on FreeBSD 9. I have know idea how is this possible,but today he report me that it is almost done. I will stay in touch to inform the [H] when its completely done. Just for the curios.
 
nl-darklord said:
the 2nd nic is not supported by default in esxi.

i haven't tried it yet, but this should work:
http://lime-technology.com/forum/index.php?topic=17936.0
Click to expand...

Thanks, managed to install the second nic in ESXi!

Only problem I'm still having is that I can' t connect to my VM server with Napp-it after Napp-it installation on the OI VM !
tried serverip:81 but no luck. I can see the server attached to my router and I can update it so it's attached!

EDIT: Seems to work now,managed to use napp-it!

EDIT 2 : Ok, I managed to get my 1st VM of OI+Napp-it working and imported my 2 pools!
 
Last edited:
siamsquare said:
Thanks, managed to install the second nic in ESXi!

Only problem I'm still having is that I can' t connect to my VM server with Napp-it after Napp-it installation on the OI VM !
tried serverip:81 but no luck. I can see the server attached to my router and I can update it so it's attached!
Click to expand...

Have you replaced serverip with your ip like
http://192.168.1.1:81

If you have a live-version, you can also start a browser on OI and enter
http://localhost:81


You can also open a console and enter
ps axw | grep mhttpd
to check if the minihttpd webserver (currently used by napp-it) is running
 
_Gea said:
Have you replaced serverip with your ip like
http://192.168.1.1:81

If you have a live-version, you can also start a browser on OI and enter
http://localhost:81


You can also open a console and enter
ps axw | grep mhttpd
to check if the minihttpd webserver (currently used by napp-it) is running
Click to expand...

Found it! Seems that after the install of the drivers of my second NIC I could connect to Napp-it!
 
Have a question about ZFS NAS or any NAS in general. Would it help performance to add additional network cards? I wanted to setup a NAS to host virtual machines as well as file storage for other computers.

Would there be the ability to assign certain network cards within the NAS server to only handle traffic related to virtual machines and if I were doing PC backups or copying data to use a different network card?
 
I have two raidz2 vdevs. vdev1 is 6 2TB Hitachi 53K000 and vdev2 is 6 2TB Hitachi 73K000. I also have a 2TB Hitachi 73K000 spare.

One of the disks in vdev1 faulted on the spare was set in use and the pool is degraded.
I pulled out the 'faulty' HD and ran DFT on it and it passed without errors. When I put the drive back, the drive is set to UNAVAIL and cannot open. I rebooted and there are no issues with the controllers (2xM1015). It doesnt appear to be a bad drive.

Any guidance is appreciated.

Code: 
pool: zfs_p1
 state: DEGRADED
status: One or more devices could not be opened.  Sufficient replicas exist for
	the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
   see: http://illumos.org/msg/ZFS-8000-2Q
  scan: resilvered 1.44T in 10h24m with 0 errors on Wed Nov 21 07:40:58 2012
config:

	NAME                         STATE     READ WRITE CKSUM
	zfs_p1                       DEGRADED     0     0     0
	  raidz2-0                   DEGRADED     0     0     0
	    c2t5000CCA369CAB2A4d0    ONLINE       0     0     0
	    c2t5000CCA369CD3F3Ad0    ONLINE       0     0     0
	    spare-2                  DEGRADED     0     0     0
	      c2t5000CCA369CD615Fd0  UNAVAIL      0     0     0  cannot open
	      c2t5000CCA36AC2ADB6d0  ONLINE       0     0     0
	    c2t5000CCA369CD6427d0    ONLINE       0     0     0
	    c2t5000CCA369CF89D1d0    ONLINE       0     0     0
	    c2t5000CCA369CF9DD7d0    ONLINE       0     0     0
	  raidz2-1                   ONLINE       0     0     0
	    c2t5000CCA36AC2A785d0    ONLINE       0     0     0
	    c2t5000CCA36AC2ADB2d0    ONLINE       0     0     0
	    c2t5000CCA36AC2BCCAd0    ONLINE       0     0     0
	    c2t5000CCA36AC2BCCDd0    ONLINE       0     0     0
	    c2t5000CCA36AC2BCDAd0    ONLINE       0     0     0
	    c2t5000CCA36AC2D771d0    ONLINE       0     0     0
	spares
	  c2t5000CCA36AC2ADB6d0      INUSE     currently in use

errors: No known data errors
 
try a disk replace
zpool replace zfs_p1 c2t5000CCA36AC2ADB6d0 c2t5000CCA369CD615Fd0

or a clear error
zpool clear zfs_p1
 
NCFireRescu said:
Have a question about ZFS NAS or any NAS in general. Would it help performance to add additional network cards? I wanted to setup a NAS to host virtual machines as well as file storage for other computers.

Would there be the ability to assign certain network cards within the NAS server to only handle traffic related to virtual machines and if I were doing PC backups or copying data to use a different network card?
Click to expand...

Usually you separate traffic based on the ip of the cards
 
I exported a zpool which had an iscsi, and now that i imported it back in, how do i re-enable the iscsi from it again?

thanks

i tried running the following to recreate the lu with the same guid, but it complains about the guid length, even though im sure it is right, 32 hex characters.

admin@openindiana:~# stmfadm create-lu -p guid=600144F04B410A0000005063D8FB001 /storage/iscsi/data
stmfadm: unable to set guid: invalid length
 
Last edited:
bbzidane said:
I exported a zpool which had an iscsi, and now that i imported it back in, how do i re-enable the iscsi from it again?

thanks

i tried running the following to recreate the lu with the same guid, but it complains about the guid length, even though im sure it is right, 32 hex characters.

admin@openindiana:~# stmfadm create-lu -p guid=600144F04B410A0000005063D8FB001 /storage/iscsi/data
stmfadm: unable to set guid: invalid length
Click to expand...

Try
stmfadm import-lu (if file, add /path/file) or
stmfadm import-lu (if volume,add /dev/zvol/rdsk/..)

in napp-it, use menu Comstar - Logical units - Import LU
then recreate a target, a target group and a view
 
that did the trick
thanks

fyi, spotted a typo
when creating a target
"Succeffully create target with alias name : 09.26.2012"


_Gea said:
Try
stmfadm import-lu (if file, add /path/file) or
stmfadm import-lu (if volume,add /dev/zvol/rdsk/..)

in napp-it, use menu Comstar - Logical units - Import LU
then recreate a target, a target group and a view
Click to expand...
 
So after fighting with OpenSolars/Solaris 11 for way too long I ended up having multiple problems. For starters there was a silently bad hard drive causing problems and even after replacing that and bringing my network down to a cross over cable I was still seeing major latency issues due to TCP retransmissions. Between that and reading about the direction of Solaris vs OpenIndiana I decided to completely wipe clean and reinstall with OpenIndiana.

I've reinstalled and so far with the cross-over cable I'm not seeing the latency. I'm going to reintroduce each bit of complexity one at a time to verify it does not return. However now I'm wondering what my end goal should be for the networking. I have 4 x GB NICs for iSCSI traffic and 2 x GB switches. The goal is to have redundancy with the most bandwidth for VMware shared storage. Obviously I should split my NICs between the switches for redundancy. Should I just use each NIC individually and let VMware handle the multipathing (4 paths)? Should I have 2 x 2 with aggregation (1 for each switch - 2 paths). I know Solaris supported IPMP, does OpenIndiana? If so, should I use that? Thanks for your insight.
 
_Gea said:
try a disk replace
zpool replace zfs_p1 c2t5000CCA36AC2ADB6d0 c2t5000CCA369CD615Fd0

or a clear error
zpool clear zfs_p1
Click to expand...

Well I really screwed myself. Prior to reading your suggestion, I had read a post that there was a known bug and to run the command:
zpool replace zfs_p1 c2t5000CCA369CD615Fd0 c2t5000CCA369CD615Fd0

Now I get this:
Code: 
NAME                               STATE     READ WRITE CKSUM     CAP            Product
	zfs_p1                             DEGRADED     0     0     0
	  raidz2-0                         DEGRADED     0     0     0
	    c2t5000CCA369CAB2A4d0          ONLINE       0     0     0     2.00 TB        Hitachi HDS5C302
	    c2t5000CCA369CD3F3Ad0          ONLINE       0     0     0     2.00 TB        Hitachi HDS5C302
	    spare-2                        DEGRADED     0     0     0
	      replacing-0                  UNAVAIL      0     0     0  insufficient replicas
	        c2t5000CCA369CD615Fd0/old  UNAVAIL      0     0     0  cannot open                    
	        c2t5000CCA369CD615Fd0      UNAVAIL      0     0     0  cannot open                    
	      c2t5000CCA36AC2ADB6d0        ONLINE       0     0     0     2.00 TB        Hitachi HDS72302
	    c2t5000CCA369CD6427d0          ONLINE       0     0     0     2.00 TB        Hitachi HDS5C302
	    c2t5000CCA369CF89D1d0          ONLINE       0     0     0     2.00 TB        Hitachi HDS5C302
	    c2t5000CCA369CF9DD7d0          ONLINE       0     0     0     2.00 TB        Hitachi HDS5C302
	  raidz2-1                         ONLINE       0     0     0
	    c2t5000CCA36AC2A785d0          ONLINE       0     0     0     2.00 TB        Hitachi HDS72302
	    c2t5000CCA36AC2ADB2d0          ONLINE       0     0     0     2.00 TB        Hitachi HDS72302
	    c2t5000CCA36AC2BCCAd0          ONLINE       0     0     0     2.00 TB        Hitachi HDS72302
	    c2t5000CCA36AC2BCCDd0          ONLINE       0     0     0     2.00 TB        Hitachi HDS72302
	    c2t5000CCA36AC2BCDAd0          ONLINE       0     0     0     2.00 TB        Hitachi HDS72302
	    c2t5000CCA36AC2D771d0          ONLINE       0     0     0     2.00 TB        Hitachi HDS72302
	spares
	  c2t5000CCA36AC2ADB6d0            INUSE     currently in use     2.00 TB        Hitachi HDS72302

I have run a scrub and a zpool clear and rebooted. Not sure what to do now. HELP!!
 
Everything seems like it should be.
-You have tried to replace a missing disk with a missing disk, resulting in an error
-You have a Spare that replaced this faulted disk (Spare is now in use)

Needed Actions:
Insert a working disk and do a
zpool replace c2t5000CCA369CD615Fd0 'new and working disk'
A resilver will start and after that your spare is available again

-Clear remaining errors
 
You must log in or register to reply here.
Back
Top