OpenSolaris derived ZFS NAS/ SAN (OmniOS, OpenIndiana, Solaris and napp-it)

Ah, maybe the docs are just confusing then. I took 'not visible' to mean, well, not visible :) Like I (think) I said, using zfsguru, if the initiatorname doesn't match, the client can't even see the target. I guess it is not a big deal - my motivation was partly to avoid confusion and partly to avoid the outside chance of hosing some other target by entering the wrong one... Thanks!
 
AFAIK FreeBSD and Solaris use different types of iSCSI. Solaris uses comstar and FreeBSD uses istgt. I guess they treat host access differently.

Matej
 
Yeah, maybe so. If so, I think the solaris way is kind of broken. Either that or the docs should be changed. YMMV of course :)
 
levak said:
Another thing: how do I set MAILFROM in nappit? Not it's sending the email as [email protected] which my mail server wont allow to relay. I want to change it to [email protected]. Where can I set that?

Thanks, Matej
Click to expand...


napp-it 0.416 nightly is available

changelog:
-napp-it setup: mailfrom setable (recreate email jobs needed)
-up from now, i will start localisation to other languages (currently en+de,
others with community help)

Gea
 
Last edited:
_Gea said:
napp-it 0.416 nightly is available

changelog:
-napp-it setup: mailfrom setable (recreate email jobs needed)
-up from now, i will start localisation to other languages (currently en+de,
others with community help)

Gea
Click to expand...

Great! Now I can use email crontab:)

Thanks,Matej
 
I have finally found a solution to how to mount a snapshot of filebased LU as another LU and get the files I needed (instead of rolling back the while snapshot, which sometimes is not an option).

So, here are the steps:)
Lets say filebased LUs are located in zpool1/iscsi and in there, there is a filebased lu iscsi-lu and we have a zfs folder snapshot called zpool1/iscsi@yesterday.

We can't just create a LU directly from a snapshot like I tried(system won't allow it). Instead, we have to clone the snapshot and make ZFS folder out of it. We do it with the following command: zfs clone source destination
zfs clone zpool1/iscsi@yesterday zpool1/iscsi-clone

Now we have a clone of the snapshot that system allows us to change. We create a LU(currently that is not possible from the web gui). We use the command sbdadm create-lu LU-location(where clone in located)
sbdadm create-lu /zpool1/iscsi-slone/iscsi-lu
The command creates a new LU with a new GUID. We can't use import-lu, because we can't have 2 LUs with the same GUID.

And finally, we create an iSCSI view:
stmfadm add-view -h hostgroup -t target-group GUID(which the command above gave). This step we can also do in web GUI.

Relogin to iscsi target and we can see the new drive. Juhej!:)

Matej
 
Gea: thanks for the update :) the changes to email functionality should be useful.

I've noticed several functions / sections that are present in the UI, but not yet developed. What is your plan for development? Is there an order / structure to how you will implement these functions? Do you need help?
 
Gigas-VII said:
I've noticed several functions / sections that are present in the UI, but not yet developed. What is your plan for development? Is there an order / structure to how you will implement these functions? Do you need help?
Click to expand...

I'm similarly curious. I think the potential impact of napp-it as a solution is greater than may be realized. Consider the limitations of competing options:

Nexentastor: 18TB limit, or thousands of dollars
Fishworks OS: Only available on Sun/Oracle hardware, thousands of dollars
FreeBSD-based solutions(sub.mesa, freenas, etc): Older ZFS version

So please consider kicking this dev up a notch - maybe at least put a donate link somewhere so we can help motivate things :)

Top of my wishlist: Scheduled replication using ZFS send to remote hosts
 
I'd like to see the LACP Link Aggregation options fleshed-out. Also, I think it would be good to have a dedicated web-developer working on the front-end parts of napp-it, to make the UI simple and straight-forward (and well-coded), without limiting the feature-set (advanced mode, additional input parameters, etc.). I'd love to see the WebUI on the same level as FreeNAS, pfSense, etc.

Also, maybe a page for setting share and folder ACLs within the filesystem? I know I can do this through windows but it's something immediately missing from napp-it.
 
Astronot said:
I'm similarly curious. I think the potential impact of napp-it as a solution is greater than may be realized. Consider the limitations of competing options:

Nexentastor: 18TB limit, or thousands of dollars
Fishworks OS: Only available on Sun/Oracle hardware, thousands of dollars
FreeBSD-based solutions(sub.mesa, freenas, etc): Older ZFS version

So please consider kicking this dev up a notch - maybe at least put a donate link somewhere so we can help motivate things :)

Top of my wishlist: Scheduled replication using ZFS send to remote hosts
Click to expand...

Thanks about that ideas :))
Currently napp-it is mainly a development for my own needs.
As a hobby i publish the results and add some features, i do not need by my own
like the email additions. But you are right. I'm on a break even point for a hobby project.

About replication.
Build a server group and replicate between them via autojob-replicate is already running on Nexenta.
There are already some problems with remote control on SE11/ OI.
Some work for this is already needed.

Gea
 
Gigas-VII said:
I'd like to see the LACP Link Aggregation options fleshed-out. Also, I think it would be good to have a dedicated web-developer working on the front-end parts of napp-it, to make the UI simple and straight-forward (and well-coded), without limiting the feature-set (advanced mode, additional input parameters, etc.). I'd love to see the WebUI on the same level as FreeNAS, pfSense, etc.

Also, maybe a page for setting share and folder ACLs within the filesystem? I know I can do this through windows but it's something immediately missing from napp-it.
Click to expand...


Thanks about your thoughts.
Current napp-it is release 0.4 and functionality is the first development issue.
So I will keep the UI as simple as possible from a technical view. But I made some efforts like independant
css files for each os to be prepared for further additions. I already separate content and style to be prepared
for GUI reworks.

Currently I work mainly on replication, localisation with all texts in editable language files and on improvements
with ZFS, disk and pool-managment. Improvements in network settings are not planned in the near future.
This is related to the expected changes in Nexenta (move to NexentaCore 4. and Illumos).

About link aggregation
I will not include settings in the near future if at all.
For my own, i have updated all my servers to 10 GBe. Link aggregation is a dead-end technology with a lot of problems.
Its only good for a lot of parallel transfers, not for a single one. 10 GBe is currently affordable between single computers
and I expect it to be common in near future.


Gea
 
Hey Gea,

I finally got my last hard drive up and running and used napp-it to configure my zpool, everything is working great.

I was wondering if it would be possible to enable SSL support for the napp it minihttpd?

Thanks very much,
s0rce
 
s0rce said:
Hey Gea,

I finally got my last hard drive up and running and used napp-it to configure my zpool, everything is working great.

I was wondering if it would be possible to enable SSL support for the napp it minihttpd?

Thanks very much,
s0rce
Click to expand...


it could be done
see http://acme.com/software/mini_httpd/

but there is only a minimal extra security with SSL and private certificates.
on a switched network, you need access to your switch to monitor your port
or you have to flood it to disable switch functionality.

if you can attack at such a level, its not a problem to add a keylogger or create a successful
man-in-the middle attack where ssl with private certificated are useless.

-> you should not allow napp-it, ssh or things like that on unsecure networks at all.
if you need access from internet, use vpn.

Gea
 
I just redid my server, now running Solaris Express and NAPP-IT: Athlon II at 3.3Ghz, 16GB ECC DDR3, and 15x Samsung 2TB drives (three groups of five drives in RAIDZ1). Since I haven't seen anyone posting solid numbers for encryption performance, here's what I'm seeing (local copy of multi-gigabyte files from backup->dozer):

Code: 
zpool iostat 10 output:
----------  -----  -----  -----  -----  -----  -----
backup      6.94T   315G  1.39K     17   176M  52.6K
dozer       6.31T  20.9T      0  1.44K      0   179M
rpool       10.5G   138G     22      0   961K  1.20K
----------  -----  -----  -----  -----  -----  -----
backup      6.94T   315G  1.34K     14   169M  52.8K
dozer       6.31T  20.9T      2  1.42K  3.30K   174M
rpool       10.5G   138G      4     24  35.6K   161K
----------  -----  -----  -----  -----  -----  -----
backup      6.94T   315G  1.45K     23   184M   184K
dozer       6.31T  20.9T      1  1.42K  1.80K   175M
rpool       10.5G   138G      1     29  29.2K   228K
----------  -----  -----  -----  -----  -----  -----

This puts load on the server at around 9.0-10.5, and can saturate a gig-E link. You definitely need multiple cores to make encryption viable (AES-128). With encryption off, load is a LOT lower, and I've been able to transfer in excess of 500MB/sec locally.
 
Do we know if hardware encryption for ZFS works natively with the new Xeon E3 processors? (ie through AES-NI)
 
Yes, it does, on Solaris Express 11 at least. In my case, I wanted ECC without going to a Xeon, so I bought AMD. I don't have any benchmarks of AES-NI on Intel platforms, unfortunately, but it's confirmed to work via posts on the ZFS mailing list.
 
Wuuhu my server is finally running. About 3 weeks ago I heard for nappit and solaris for the first time and gave it a try. Now I can say I understand enough of solaris and ZFS(the fact that I'm using linux for the last 10 years helped a lot) to use it as my SAN/NAS. With Gea's nappit, everything is just soooo much easier, but since it developed for his needs, sometimes you just have to open a console and so some manual work:)

Anyway, system is up and in production(juhej!).

Currently I have only 1 pool with 4x1TB drives(3x Samsung F3 and one Seagate xSeries) and bonnie gives me these numbers:
- Seq write: 160MB/s
- Seq rewrite: 104MB/s
- Per Character write: 75MB/s
- Seq read: 300MB/a
- Per Character read: 90MB/s
- Random seeks: around 2700/s

Pretty neat:)

iSCSI bench to an old P4 based Xeon:
- Seq write: 70MB/s
- Seq rewrite: 40MB/s
- Per Character write: 30MB/s
- Seq read: 104MB/a
- Per Character read: 35MB/s
- Random seeks: around 500/s

Samba copy to an intel core 2 quad (Q6600 i think) and win7 64bit gives mixed results:
Total commander: write and read above 100MB/s
Windows explorer: write and read around 65-80MB/s
DiskBench: write 12MB/s(constant), read above 100MB/s

NFS is out of the question, since it only supports sync writes and not having a good SSD ZIL, speeds are just too damn slow. Too bad, because getting files out of snapshots is just so much easier:)

Anyway, system works great and I am very satisfied with it. A lot more that I was with OpenFiler.

Thanks Gea!

Matej
 
Really? How?

Only solution I found was to disable ZIL, which I don't want...

Matej
 
new napp-it nightly 0.415c available

Changelog:
Localisation/ versions in different languages: see text-files in napp_it/zfsos/_lib/lang

If you like to have a version in your language (non english or german version, they are already on the way):

-Copy the files from /en to a folder for your language ex /sp
-Translate the files to your language, zip them and send it to [email protected]

(at least the main menus and the basic infos to start with a localized version, that can be completed step by step)



Gea
 
Last edited:
small bug in 0.416c:

in the System tab in the english napp it the subtabs are in german:

| Hilfe | Dienste | Hardware | Netzwerk | Firewall | Log | PID | Power Management | Statistik | Herunterfahren |
 
Trying to install to ESXi, when selecting the OS for the setup type what should i pick? trying to install solaris express 11 but it doesn't get anywhere. also which version should i be downloading from the website maybe thats the problem LOL.
 
tried that and it doesn't work, let me try it again and report back what it says.

EDIT, ok so i just launched the setup again and its got something to do with the networking. its trying to send packet to 224.0.0.251 and fails and eventually just goes to a console login
 
Last edited:
Ok, I'm a newb so forgive the idiotic questions.

I tried FreeNAS 8 RC5 and created pool of 3 drives in raidz for fun. I realize FreeNAS is a simple easy to use solution and I could foresee myself starting out with that. However, what if I decide to play around more? I tried OI and Napp-it and I like it very much. After all, a desktop environment suits my newbiness more. Haha. Anyway, I also realize it has a later ZFS version, so that might be a lot better.

I tried exporting the zpool in FreeNAS 8. No problem. Installed OI, but I can't seem to find any pools to import? Could this have anything to do with the fact that in FreeNAS, my drives were named ad7, ad9, ad10, ad12, whereas in OI they're named in dev/dsk/ differently? Shrug. What am I doing wrong here... lol.
 
dmo580 said:
Ok, I'm a newb so forgive the idiotic questions.

I tried FreeNAS 8 RC5 and created pool of 3 drives in raidz for fun. I realize FreeNAS is a simple easy to use solution and I could foresee myself starting out with that. However, what if I decide to play around more? I tried OI and Napp-it and I like it very much. After all, a desktop environment suits my newbiness more. Haha. Anyway, I also realize it has a later ZFS version, so that might be a lot better.

I tried exporting the zpool in FreeNAS 8. No problem. Installed OI, but I can't seem to find any pools to import? Could this have anything to do with the fact that in FreeNAS, my drives were named ad7, ad9, ad10, ad12, whereas in OI they're named in dev/dsk/ differently? Shrug. What am I doing wrong here... lol.
Click to expand...

if there are no format-specific problems and if the target system support the ZFS-version,
import is always possible.

maybee you problem is similar to
http://hardforum.com/showthread.php?t=1575034&highlight=zfsguru+import+nexenta

Gea
 
I'm having problems setting permissions in Windows.

I joined nappit server to domain and did some idmap-ing:
root@fatlady-dmz:/zpool2# idmap list
add winuser:*@pulsar unixuser:*
add "wingroup:Domain Users@pulsar" unixgroup:users
add "wingroup:Domain Admins@pulsar" unixgroup:staff
add winuser:Administrator@pulsar unixuser:root
add winuser:levak@pulsar unixuser:levak

After that, I created a "default" ACL for a ZFS folder:
/usr/bin/chmod -R A=everyone@:full_set:fd-----:allow zpool2/arhiv

I then mapped zpool2/arhiv in windows and tried to set ACLs in Properties-Security. I can already see entry Everyone with full rights.
I added user levak and gave him some permissions. When I click apply, I get an error saying:
An error occured while applying security informations to:
X:\folder
No mapping between accound name and security IDs was done.
Click to expand...

I can click Continue, but I get an error saying Unable to save permissions and options Cancel/Retry

Matej
 
levak said:
I'm having problems setting permissions in Windows.

I joined nappit server to domain and did some idmap-ing:
root@fatlady-dmz:/zpool2# idmap list
add winuser:*@pulsar unixuser:*
add "wingroup:Domain Users@pulsar" unixgroup:users
add "wingroup:Domain Admins@pulsar" unixgroup:staff
add winuser:Administrator@pulsar unixuser:root
add winuser:levak@pulsar unixuser:levak

After that, I created a "default" ACL for a ZFS folder:
/usr/bin/chmod -R A=everyone@:full_set:fd-----:allow zpool2/arhiv

I then mapped zpool2/arhiv in windows and tried to set ACLs in Properties-Security. I can already see entry Everyone with full rights.
I added user levak and gave him some permissions. When I click apply, I get an error saying:


I can click Continue, but I get an error saying Unable to save permissions and options Cancel/Retry

Matej
Click to expand...

i suggest always:
- do not set mappings unless you absolutely need them
with domains, the only sometimes usefull setting is
winuser: one domainadmin = unixuser:root
wingroup: domainadmins = unixgroup: root
- do not set ACL from Solaris unless you absolutely need it

try my way:
If you need to set file and folder ACL:
- reassign the root pw (passwd root, only needed after first install) to create a smb pw for root
- smb-connect as root and set needed ACL from Windows

if you need to set share-ACL
- smb connect from Windows as a user, member of Solaris SMB-admins
- Start Windows computer management
- connect computer management with your Solaris Server
- Set share ACL via computer management.

Gea
 
Is it okay to mix TLER and non-TLER drives? I've got 1 WD3200SD(WD Caviar SE) drive and 8 WD3200JD(WD Caviar RE) drives. Haven't added the SE drive to the pool yet, but I expect to try later today unless I hear otherwise.
 
TeeJayHoward said:
Is it okay to mix TLER and non-TLER drives? I've got 1 WD3200SD(WD Caviar SE) drive and 8 WD3200JD(WD Caviar RE) drives. Haven't added the SE drive to the pool yet, but I expect to try later today unless I hear otherwise.
Click to expand...

Raid drives with TLER are developed to use them with a hardware raid controller.
They are not needed/ unwanted with ZFS.

But it should not be a problem (I also use some WD RE4 drives without disabling TLER).


Gea
 
- reassign the root pw (passwd root, only needed after first install) to create a smb pw for root
Click to expand...

What do you mean by that? I just do 'passwd root'? Is there any idmap by default (like Administrator=root) - I flushed all idmaps.

Matej
 
Hi

Noob here. For your statement
LSI Controller based on 1068e
Click to expand...

Do these include the built in motherboard LSI controller? or better to get a standalone card, if the later can I get some recomendations to look ebay for deals?

Thanks

PSS how bad is performance on 4K format 2TB drives?? has it been benchmarked in comparison with a number, say 30% decrease in performance or anything like that? Thanks.
 
intel said:
Hi

Noob here. For your statement

Do these include the built in motherboard LSI controller? or better to get a standalone card, if the later can I get some recomendations to look ebay for deals?
Click to expand...

all the same, if you can get it internal (ex on a SuperMicro Board) its cheaper.
Mostly external versions have IT firmware, on internal versions you may flash
the firmware from Raid to IT version.

PSS how bad is performance on 4K format 2TB drives?? has it been benchmarked in comparison with a number, say 30% decrease in performance or anything like that? Thanks.
Click to expand...

see
http://digitaldj.net/2010/11/03/zfs-zpool-v28-openindiana-b147-4k-drives-and-you/

from the values:
sequential read is nearly the same, write is about 15% worser.

from my own experience with IT
Performance differences below 30% are mostly not really worth to consider.
Values above and you have a 'Yes, i can feel it'

Gea
 
_Gea said:
all the same, if you can get it internal (ex on a SuperMicro Board) its cheaper.
Mostly external versions have IT firmware, on internal versions you may flash
the firmware from Raid to IT version.



see
http://digitaldj.net/2010/11/03/zfs-zpool-v28-openindiana-b147-4k-drives-and-you/

from the values:
sequential read is nearly the same, write is about 15% worser.

from my own experience with IT
Performance differences below 30% are mostly not really worth to consider.
Values above and you have a 'Yes, i can feel it'

Gea
Click to expand...

Thank you for your quick and detailed response.

I've tried NexentaStor and OpenIndiana (had tried OpenSolaris too). My motherboard (Biostar TPower I45 ) or the video card (PCI-E ATI Radeon 3800 series) seems to act very very funky with both on live-cd (slow graphics and load times for OS) for OI and NexentaStor takes forever to boot up.

Due to unsupported (it seems to me) hardware I ended up using Ubuntu + fuse-zfs which yields an awful performance shall I add.

Just reading your website it seems like ESXi can be installed on top of my current server, and with pass-through I may be able to use a Nexenta as a VM...

Is video pass-through available on ESXi, say I wanted a VM to be Windows XP and ensure that ATI video coming from the video card pushes that VM to screen? Also any tips/notes about the hardware issue noted above? many thanks
 
Ok. Got all my components in.

Running OI b148 with napp-it.

Plugged everything in, and created a raidz2 array with 6x2TB drives. Then a few hours later I noticed it shows up as degraded where the last disk is unavailable. Uh oh. Bad drive or bad cabling? Don't know, but bad drives are scary.

btw, what am I supposed to make of the "error" column?
hderror.jpg


How do I use SMART to check? Sorry. I'm a CLI newb, so I would basically have to plug this drive into my windows system to check -.-

Code: 
 smartctl -A /dev/rdsk/c6d0s0
smartctl 5.40 2010-10-16 r3189 [i386-pc-solaris2.11] (local build)
Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net


#######################################################################
ATA command routine ata_command_interface() NOT IMPLEMENTED under Solaris.
Please contact [email protected] if
you want to help in porting smartmontools to Solaris.
#######################################################################

Smartctl: Device Read Identity Failed (not an ATA/ATAPI device)

Edit: tried a new cable, and deleted the whole pool (im still testing anyway). Readded the pool and it seemd to take a while and while the disk was accessible, the # of Hard errors went up to triple digits. I rebooted and the disk showed unavailable again. I guess it' s a bad disk? I already initiated a replacement on Amazon. But if anyone has any tips to manage this better in the future that would be appreciated :)
 
Last edited:
I would like to move my web data from iscsi to NFS, but I'm having some NFS performance issues.

Speeds are ok in both, iSCSI and NFS, but file creation/deletion on NFS are sooooooo BAD:)

On iSCSI I get:
File creation: 11000/s
File deletion: 12000/s
File read: 50000/s

NFS with sync set to disable:
File creation: 2300/s
File deletion: 1600/s
File read: 5000/s

I would expect NFS to be equal to iSCSI, since I disabled the sync option for NFS. Is it enough to just disable sync in nappit or do I have to do something else?

lp, Matej
 
dmo580 said:
Smartctl: Device Read Identity Failed (not an ATA/ATAPI device)[/code]

Edit: tried a new cable, and deleted the whole pool (im still testing anyway). Readded the pool and it seemd to take a while and while the disk was accessible, the # of Hard errors went up to triple digits. I rebooted and the disk showed unavailable again. I guess it' s a bad disk? I already initiated a replacement on Amazon. But if anyone has any tips to manage this better in the future that would be appreciated :)
Click to expand...

1.
smartctrl mostly not working with onboard sata/ide

2.
i would inplug the drive and test it with a test-tool from the drive manufacturer
on a Windows or DOS (ex booted from a usb stick) machine; depends on the test-tool.

Gea
 
You must log in or register to reply here.
Back
Top