OpenSolaris derived ZFS NAS/ SAN (OmniOS, OpenIndiana, Solaris and napp-it)

just for comparison:
SAMBA has many features comparing with opensolaris ZFS builtin CIFS.

If anyone needs does not involve complicated/mix environment, opensolaris ZFS builtin CIFS is more than adequate

and there are some differences as mention on previous posting.
SAMBA need to be configure precisely :) but powerful. this is a headache at during configuration only.

opensolaris ZFS builtin CIFS is very simple and not hard to configure :)
 
Has anyone run benchmarks on a small office load on a kernel CIFS vs a Samba install? I'd be curious just how much better/worse they fare against each other in a 10-25 client load running typical office apps (as in, intermittent document access not databases.
 
newbie question regarding zfs, if i export my zpool, reinstall openindiana, and then import my zpool again, will my shares be intact?


thanks
 
bbzidane said:
newbie question regarding zfs, if i export my zpool, reinstall openindiana, and then import my zpool again, will my shares be intact?


thanks
Click to expand...

CIFS: yes (ZFS property)
NFS: yes (ZFS property)

iSCSI: no, you must import LU and recreate targets, target groups and views
netatalk, SAMBA SMB: no, you must recreate shares
 
Hi all-
I'm having some permission problems. I have a ZFS folder shared over SMB using napp-it. I added these permissions to a subfolder of the parent share:

chmod A+user:myusername:full_set:file_inherit/dir_inherit:allow subfolder

Then from my Windows 7 machine I browsed to the machine and mapped subfolder to a drive Z: logging in with the myusername and password credentials.

I am having two problems:
1) I tried installing an application to the Z: drive, (Diablo 3 Public Test), and I get the error "The folder you selected cannot be used. Please select a different folder."
2) I am able to actually select the share instead of the Z drive, ie: \\SOLARIS\home\subfolder and install there, however during installation some of the subfolders created don't seem to correctly inherit the parent settings I specified:

Code: 
d---------+  2 myusername staff          2 Sep 23 22:31 Updates
       group:2147483654:rwxpdDaARWcCos:fd-----:allow
       group:2147483655:rwxpdDaARWcCos:fd-----:allow
        user:2147483648:rwxpdDaARWcCos:fd-----:allow

Which ends up breaking the installer saying it can't read important files.

Also if this is relevant, it asks for UAC permission before installing.

Does anyone have any suggestions on what could be going on here, or how to fix?

Thanks!
 
wkearney99 said:
Has anyone run benchmarks on a small office load on a kernel CIFS vs a Samba install? I'd be curious just how much better/worse they fare against each other in a 10-25 client load running typical office apps (as in, intermittent document access not databases.
Click to expand...

since no one reply,
kernel CIFS should better in theoritically :) since no additional layers are involved, especially with built-in ZFS
BUT....
with modern technology, the differences is negligible due to fast processing on current technology (CPU, I/O, and others).

as Mine:
I switched from OI ZFS (NFS and CIFS) to centos 6.3 with zfsonlinux RC11 ( NFS and CIFS/SMB), and see better performance on NFS ( SAMBA CIFS mostly identical performance on my testing) as an "other" backup server for vmware and files(some big/small file sized). this is an old lga771 with SAT2-MV8 :) with 16G memory.yeah heat maker-> FB-DIMM hehehe
 
halcyon said:
Hi all-
I'm having some permission problems. I have a ZFS folder shared over SMB using napp-it. I added these permissions to a subfolder of the parent share:

chmod A+user:myusername:full_set:file_inherit/dir_inherit:allow subfolder

Then from my Windows 7 machine I browsed to the machine and mapped subfolder to a drive Z: logging in with the myusername and password credentials.

I am having two problems:
1) I tried installing an application to the Z: drive, (Diablo 3 Public Test), and I get the error "The folder you selected cannot be used. Please select a different folder."
2) I am able to actually select the share instead of the Z drive, ie: \\SOLARIS\home\subfolder and install there, however during installation some of the subfolders created don't seem to correctly inherit the parent settings I specified:

Code: 
d---------+  2 myusername staff          2 Sep 23 22:31 Updates
       group:2147483654:rwxpdDaARWcCos:fd-----:allow
       group:2147483655:rwxpdDaARWcCos:fd-----:allow
        user:2147483648:rwxpdDaARWcCos:fd-----:allow

Which ends up breaking the installer saying it can't read important files.

Also if this is relevant, it asks for UAC permission before installing.

Does anyone have any suggestions on what could be going on here, or how to fix?

Thanks!
Click to expand...

Mostly this is a problem with mixed Unix and ACL permissions where you have
the problem, that setting Unix permissions is deleting ACL and ACL inheritance.
(CIFS is ACL only and other Unix services keeps attention to ACL as well)

I would start with a new ZFS dataset
- Set/Keep Unix permissions to 777 and restrict access with ACL only
(do not set Unix permissions afterwards or CIFS gets problems)

- set ACL of this dataset folder to root =full and a special user or everyone@ =modify
with file and folder inheritance=on

After this, all newly created files and folders inherits this ACL


current problem - possible reasons:
- Setting ACL does not affect already created files (you must reset recursively)
- mixed ACL and Unix permissions where setting Unix permissions deletes ACL and ACL inheritance
- too restrictive Unix permissions (I usually start with Unix 777 and restrict effective permissions via ACL only when using CIFS)

you may use napp-it ACL extension to
- reset Unix permission recursively to 777 (or you may not be able to set ACL's)
- reset ACL recursively to roor=full and a user or everyone@=modify to restrict effective permissions
 
Ok so setting 777 on the directory allowed D3 to 'pick' that folder for installation (but this is an unusable permissions mask for me when I share this over NFS so something must be changed). However I still ran in to the installer mucking around with permissions causing it to not inherit correctly and fail installation. Here are the permissions:

Code: 
myusername@solaris:/main/home$ ls -V
total 5
drwxrwxrwx+  3 myusername staff          3 Sep 24 10:04 david
              user:root:rwxpdDaARWcCos:fd-----:allow
          user:myusername:rwxpdDaARWcCos:fd-----:allow
                 owner@:rwxp-DaARWcCos:-------:allow
                 group@:rwxp-DaARWc--s:-------:allow
              everyone@:rwxp-DaARWc--s:-------:allow

myusername@solaris:/main/home$ cd david
myusername@solaris:/main/home/david$ ls -V
total 5
drwx------+  7 myusername staff         25 Sep 24 10:04 Diablo III Public Test
       group:2147483655:rwxpdDaARWcCos:fd-----:allow
       group:2147483654:rwxpdDaARWcCos:fd-----:allow
       group:2147483656:rwxpdDaARWcCos:fd-----:allow
          user:myusername:rwxpdDaARWcCos:-------:allow
        user:2147483648:rwxpdDaARWcCos:fdi----:allow
              user:root:rwxpdDaARWcCos:fd----I:allow
          user:myusername:rwxpdDaARWcCos:fd----I:allow

myusername@solaris:/main/home/david$ cd Diablo\ III\ Public\ Test/
myusername@solaris:/main/home/david/Diablo III Public Test$ ls -V

.. snip .. 
drwx------+  3 myusername staff          3 Sep 24 10:04 Data_D3
       group:2147483655:rwxpdDaARWcCos:fd-----:allow
       group:2147483654:rwxpdDaARWcCos:fd-----:allow
       group:2147483656:rwxpdDaARWcCos:fd-----:allow
          user:myusername:rwxpdDaARWcCos:-------:allow
        user:2147483648:rwxpdDaARWcCos:fdi----:allow
       group:2147483655:rwxpdDaARWcCos:fd----I:allow
       group:2147483654:rwxpdDaARWcCos:fd----I:allow
       group:2147483656:rwxpdDaARWcCos:fd----I:allow
          user:myusername:rwxpdDaARWcCos:------I:allow
        user:2147483648:rwxpdDaARWcCos:fdi---I:allow
              user:root:rwxpdDaARWcCos:fd----I:allow
          user:myusername:rwxpdDaARWcCos:fdi---I:allow

d---------+  2 myusername staff          2 Sep 24 10:04 Updates
       group:2147483654:rwxpdDaARWcCos:fd-----:allow
       group:2147483655:rwxpdDaARWcCos:fd-----:allow
        user:2147483648:rwxpdDaARWcCos:fd-----:allow

As you can see there are two different subdirectories of D3 Public Test with wildly different permissions, which really confuses me, both in terms of unix permissions and ACLs. I took a look to see what those numbers corresponded to, and on the Updates folder (which gets the error) Windows thinks the groups are Administrators (SOLARIS\Administrators) and Users (SOLARIS\Users), and the user is CREATOR OWNER.

For the folder I can read correctly it gets a ton of permissions according to Windows, and evidenced by the above list of ACLs: CREATOR OWNER, Account Unknown(S something something), myusername (SOLARIS\myusername), Administrators (SOLARIS\Administrators), Users (SOLARIS\Users), INTERACTIVE.

I checked my local D3 install, and permissions on both folders are CREATOR OWNER, Administrators (MAIN\Administrators) [MAIN is my local machine's name], and Users (MAIN\Users).

I guess one thing I don't understand is what the Administrators or Users group is on SOLARIS since that machine is not Windows, and probably doesn't even have that group. Do I somehow need to create the Administrators/Users group on the Solaris machine, and add my username to that?
 
Last edited:
halcyon said:
I checked my local D3 install, and permissions on both folders are CREATOR OWNER, Administrators (MAIN\Administrators) [MAIN is my local machine's name], and Users (MAIN\Users).

I guess one thing I don't understand is what the Administrators or Users group is on SOLARIS since that machine is not Windows, and probably doesn't even have that group. Do I somehow need to create the Administrators/Users group on the Solaris machine, and add my username to that?
Click to expand...

If you are coming from the Unix/Linux world, you must accept that Solaris CIFS is mostly like Windows. It uses Windows SIDs, ACL and Windows User and Group management (users are the same like on Unix but groups are different) with the same default users and groups like on Windows (see napp-it menu user)

If you add this with NFS3 you have the problem that you want to combine completely incompatible approaches

What I do.
- Use only ACLs (They are valid for all services, even NFS)
- set aclinherit and aclmode to passthrough (Windows like, napp-it default)
- After creation of a folder, I reduce the permissions to a minimum like
root=full, everyone or a user=modify or everyone=create without inheritance and owner=full

Use ACL everyone@ (or user nobody) for NFS and set NFS access based on IP

and avoid mixed NFS/CIFS shares when not absolutely needed
You should also set ACLs either from Windows (as user root) or napp-it ACL extension.
Its nearly impossible to handle ACL via CLI.
 
Last edited:
I guess part of the problem is I am pretty unfamiliar with Windows ACL/file security model.

_Gea said:
- set aclinherit and aclmode to passthrough (Windows like, napp-it default)
Click to expand...

Where should I set this after a folder is created?

- After creation of a folder, I reduce the permissions to a minimum like
root=full, everyone or a user=modify or everyone=create without inheritance and owner=full
Click to expand...

So something like:
chown root:root folder
chmod A+user:root:full_access:file_inherit/dir_inherit:allow
chmod A+user:someuser:modify:no_inherit:allow
How do you add permissions for file creator/owner?

(Excuse any errors in above I'm at work).

Also what I don't understand is that when I created those files they were referencing groups I had never created (SOLARIS\Administrators, SOLARIS\Users) and CREATOR OWNER. Do I need to do something to create these groups? And do I need to do something to put my user account (on the Solaris machine) into these groups? Because I was getting access denied even trying to browse into these folders.

Also I don't understand how the unix permissions of this particular folder became 000... what is the process for how Unix permissions on files are created when creating them from Windows?
 
So I did some more playing around with this, I believe what is happening is that Diablo 3 is explicitly changing the permissions on the files to be owned by the groups listed above, and not the owner. And the problem with ephemeral ids is that these are just created from scratch, and nothing maps the ephemeral group id's for Administrators/Users back to the user I am authenticating with, hence permission denied. I added the mapping of the Administrators group to the staff unix group, ie:

idmap add wingroup:Administrators@BUILTIN unixgroup:staff

and now that same ls -V looks like:

Code: 
d---rws---+  2 myusername staff          4 Sep 24 20:43 Updates
       group:2147483654:rwxpdDaARWcCos:fd-----:allow
            group:staff:rwxpdDaARWcCos:fd-----:allow
        user:2147483648:rwxpdDaARWcCos:fd-----:allow

And the installer and explorer can access the folder, which is good. I'm still not clear on how the unix permissions (non ACLs) are created from the ACLs, but I'll leave that mystery for another day.
 
halcyon said:
Also what I don't understand is that when I created those files they were referencing groups I had never created (SOLARIS\Administrators, SOLARIS\Users) and CREATOR OWNER. Do I need to do something to create these groups? And do I need to do something to put my user account (on the Solaris machine) into these groups? Because I was getting access denied even trying to browse into these folders.
[/QUOTE

These are default Windows groups.
Look at Solaris SMB user and group management (napp-it menu user)

[QUOTE
Also I don't understand how the unix permissions of this particular folder became 000... what is the process for how Unix permissions on files are created when creating them from Windows?
Click to expand...

When you set ACLs to files and folders they are used and not the unix permissions.
And the ACL thing is not a CIFS only mechanism but can be globally used. (CIFS is only special in always using ACL)

example
d---rws---+ 2 myusername staff 4 Sep 24 20:43 Updates
(the + after the Unix permissions indicates that there are ACLs)

The Unix permissions changed to a level that includes the ACL.
But you can ignore the Unix permission but must look at ACL

If these ACLs are set from CIFS or other Unix processes dies not matter.
The 000 permission indicates mostly, that there is no read/execute ACL for everyone.
 
Last edited:
halcyon said:
S

And the installer and explorer can access the folder, which is good. I'm still not clear on how the unix permissions (non ACLs) are created from the ACLs, but I'll leave that mystery for another day.
Click to expand...


The problem:
Unix permissions are quite simple but very limited. You only have
owner, group and everyone with read, write and execute permissions

NFS4 ACL extends this with many users, many groups, detailed inheritance options
and a much wider set or permissions like add or delete only. For compatibility reasons,
there are trivial ACL like owner@, group@ and everyone@. They behave similar to old Unix permissions but with ACL inheritance.

So there can*t be a simple mapping permissions vs ACL and there is no option to use either. When ACL are set, they are relevant and not the Unix permissions.
(You may think of a: the more restrictive is used)
 
anyone,

is there a way to move all data from a pool to a zfs folder? When initially setting up my zfs SAN I neglected to create a ZFS folder now I am unable to properly set ACL's. I need to move all data from the raid-z pool to a folder I plan on creating so that the mountpoint will be /raid-z/media. Here is the output of my session:
[?1034halex@openindiana:~# zfs list
NAME USED AVAIL REFER MOUNTPOINT
raid-z 2.49T 2.79T 2.49T /raid-z
rpool 10.0G 1.75G 47.5K /rpool
rpool/ROOT 5.78G 1.75G 31K legacy
rpool/ROOT/napp-it-0.8g_update_04.30 165K 1.75G 3.23G /
rpool/ROOT/napp-it-0.8h_update_07.14 4.83G 1.75G 3.93G /
rpool/ROOT/openindiana 13.7M 1.75G 3.23G /
rpool/ROOT/openindiana-1 936M 1.75G 3.20G /
rpool/ROOT/openindiana-2 8.99M 1.75G 3.24G /
rpool/ROOT/openindiana-3 14.0M 1.75G 3.86G /
rpool/ROOT/pre_napp-it-0.8g 139K 1.75G 3.20G /
rpool/ROOT/pre_napp-it-0.8h 70K 1.75G 3.85G /
rpool/dump 1.34G 1.75G 1.34G -
rpool/export 18.3M 1.75G 32K /export
rpool/export/home 18.3M 1.75G 32K /export/home
rpool/export/home/alex 18.2M 1.75G 18.2M /export/home/alex
rpool/swap 2.85G 4.47G 137M -
alex@openindiana:~#
 
moto316 said:
anyone,

is there a way to move all data from a pool to a zfs folder? When initially setting up my zfs SAN I neglected to create a ZFS folder now I am unable to properly set ACL's. I need to move all data from the raid-z pool to a folder I plan on creating so that the mountpoint will be /raid-z/media. Here is the output of my session:
[?1034halex@openindiana:~# zfs list
NAME USED AVAIL REFER MOUNTPOINT
raid-z 2.49T 2.79T 2.49T /raid-z
rpool 10.0G 1.75G 47.5K /rpool
rpool/ROOT 5.78G 1.75G 31K legacy
rpool/ROOT/napp-it-0.8g_update_04.30 165K 1.75G 3.23G /
rpool/ROOT/napp-it-0.8h_update_07.14 4.83G 1.75G 3.93G /
rpool/ROOT/openindiana 13.7M 1.75G 3.23G /
rpool/ROOT/openindiana-1 936M 1.75G 3.20G /
rpool/ROOT/openindiana-2 8.99M 1.75G 3.24G /
rpool/ROOT/openindiana-3 14.0M 1.75G 3.86G /
rpool/ROOT/pre_napp-it-0.8g 139K 1.75G 3.20G /
rpool/ROOT/pre_napp-it-0.8h 70K 1.75G 3.85G /
rpool/dump 1.34G 1.75G 1.34G -
rpool/export 18.3M 1.75G 32K /export
rpool/export/home 18.3M 1.75G 32K /export/home
rpool/export/home/alex 18.2M 1.75G 18.2M /export/home/alex
rpool/swap 2.85G 4.47G 137M -
alex@openindiana:~#
Click to expand...

create a dataset raid-z/media

open a shell
enter su to get superuser permission

enter nautilus (use OI live GUI filebrowser)
or mc (use CLI based Midnight commander file browser)
or use mv (Unix CLI)
 
I am trying to install illumian for a friend and right when I boot into the fresh installation, I am not able to update the system. The "apt-get update" fails. There is internet connectivity (confirmed with other sites).

When I ping apt.illumian.org it fails. Even the napp-it script does not work.

Does anybody know anything about this?

Thanks

Vangelis
 
Hey!

I just build a myself a new storage with 2 hard drives in mirror. I also have a 1TB external drive, that I would use for a backup. What is the best way to make incremental backup? Should I get some backup script or use rsync or can I also do it with zfs send/receive? How do you usually do it?

Matej
 
I'm thinking of rebuilding my nappit SAN/NAS to all-in-one. What is currently the best motherboard + CPU mix? I'm looking more for a cheap solution.

I'll need two pcie x8 slots for SAS HBA's, 2 ethernet cards, it would be nice to have IPMI as well.
On the server, I'll be running nappit SAN, 1x windows 2008 domain/printer server which idles most of the time and 1x linux server that serves as a web/mail server.

Is Supermicro X8SIL-F still the best choice? How about some Ivy Bridge mobo and cpus?

Matej
 
levak said:
Hey!

I just build a myself a new storage with 2 hard drives in mirror. I also have a 1TB external drive, that I would use for a backup. What is the best way to make incremental backup? Should I get some backup script or use rsync or can I also do it with zfs send/receive? How do you usually do it?

Matej
Click to expand...

you described the options:
rsync: keep folders in sync, cannot copy ZFS nfs4 ACL, slow with a lot of files
other tools like Windows robocopy: similar to rsync but can copy ACL

zfs send: keeps the dataset in sync with a replicated datase with all properties,
fast but copies also snapshots (may need a lot of space), works with open files
 
I've got hold of 3 of these, with the ultimate intention to attach 24 SATA drives, and looking for some words of wisdom from those who may have gone before.

2 of them are used (snagged off eBay), and the 3 cards were manufactured at approx 1 year intervals.

Since the Board name (and assembly code, if not the revision) is the same in each case, I'm wondering if I can quit worrying about the FRU being different for the 2012 card. Also, will the revisions matter? Am I worrying too much, and can I just treat them all as the same product?

FRU ; Board name ; Assembly/Rev ; Date of Manufacture
46C8933 SAS9220-8i H3-25097-03A 22/06/2012
46M0861 SAS9220-8i H3-25097-02C 02/05/2011
46M0861 SAS9220-8i H3-25097-01D 27/06/2010

What I want to do is flash all to IT mode for ZFS use. Should I treat them all as LSI 9240-8i, and flash to LSI9211-IT mode?

Finally, in recent hardware recommendations on napp-it.org _Gea suggests SuperMicro X9xxx-F mainboards with 202, 3420 or 5520 chipsets. The only m/b I can see with >=3 PCI-E x8 slots (for the aforementioned HBAs) is the X9SRL-F, but this has the C602 chipset. Has anyone had any success using this board for a ZFS file server? Should I not worry about them all being in x8 slots? If they are in a mix of x8 & x4 slots, is there some way to force all the HBAs to x4, or am I just thinking too hard about all of this?

Thanks for any input.
 
neiljt said:
Finally, in recent hardware recommendations on napp-it.org _Gea suggests SuperMicro X9xxx-F mainboards with 202, 3420 or 5520 chipsets. The only m/b I can see with >=3 PCI-E x8 slots (for the aforementioned HBAs) is the X9SRL-F, but this has the C602 chipset. Has anyone had any success using this board for a ZFS file server? Should I not worry about them all being in x8 slots? If they are in a mix of x8 & x4 slots, is there some way to force all the HBAs to x4, or am I just thinking too hard about all of this?

Thanks for any input.
Click to expand...

- I use a X9SRL-F as AV-Filer for OpenIndiana and have not had problems yet.
- Flash your HBA to LSI 9211 IT
- Do not care about PCI 4x in 8x slots - fast enough -
the slots are PCI-e 2.x and 3.x, fast enough even for next generation of HBA
 
neiljt said:
I've got hold of 3 of these, with the ultimate intention to attach 24 SATA drives, and looking for some words of wisdom from those who may have gone before.

[...]

Finally, in recent hardware recommendations on napp-it.org _Gea suggests SuperMicro X9xxx-F mainboards with 202, 3420 or 5520 chipsets.
Click to expand...

...there are reports out with the X9SCM-F with IvyBridge CPUs, where passthrough of 3 M1015 would not work.
With SandyBridge CPUs all is fine. Also reported are effects/problems with other card types when using V2.0a (IvyBridge) BIOS, even when on SB-CPUs.
See: http://lime-technology.com/forum/index.php?topic=22327.msg201887#msg201887

If you are planning to go all-in-one, be aware of that problem.
 
Gea, just wanted to post and say I've been using NAPP-IT for about 18 months now. After getting it set up, it just *works* - I haven't had to touch the box for months at a time (and this is a fairly large setup with 20x2TB drives). Appreciate all the effort you put into building this very useful piece of software!
 
After a brief play with Win Server 2012 I went back to OpenIndiana + NappIt, RAID-Z1 with 4 x 2TB Samsung 2TB HD204UI, however when writing large amounts I'm getting speed fluctuations from around 60MB/s to 0MB/s (very occasionally peaks around 80MB/s), it spends most of its time much lower, it will often slow right down to 0, then about 5 seconds later it speeds back up to around 50-60MB/s, then slows back down again, sometimes it doesn't go as low as 0b/s and copies at around 500kB/s before speeding up again. Probably averages 15-20MB/s overall.

Read speeds are fine (around 80MB/s consistent over gigabit), no compression/dedup, copying from Windows 8 via SMB share, the OI VM is running on ESXi 5.1 via a passed through M1015, 8GB RAM, Xeon E3-1230. Have run OI previously without any issues.

Anyone have any ideas of what I could check?
 
5loth said:
After a brief play with Win Server 2012 I went back to OpenIndiana + NappIt, RAID-Z1 with 4 x 2TB Samsung 2TB HD204UI, however when writing large amounts I'm getting speed fluctuations from around 60MB/s to 0MB/s (very occasionally peaks around 80MB/s), it spends most of its time much lower, it will often slow right down to 0, then about 5 seconds later it speeds back up to around 50-60MB/s, then slows back down again, sometimes it doesn't go as low as 0b/s and copies at around 500kB/s before speeding up again. Probably averages 15-20MB/s overall.

Read speeds are fine (around 80MB/s consistent over gigabit), no compression/dedup, copying from Windows 8 via SMB share, the OI VM is running on ESXi 5.1 via a passed through M1015, 8GB RAM, Xeon E3-1230. Have run OI previously without any issues.

Anyone have any ideas of what I could check?
Click to expand...

I see the same thing with a similar config. OpenIndiana with NappIt on ESXi (5GB RAM allocated), RAID-Z1 5x2TB Seagate Green drives (mix of ST32000542AS and ST2000DL003). I get solid 1Gbps line speed for about 30 seconds before the chugging starts. Reads are fine.

The same hardware in a bare metal WHS 2011 installation was much faster and consistent. I stopped trying to find a problem and decided 20MB/sec is good enough for bulk home WORM storage. Give me snapshots and data integrity over raw speed any day!

I'm using standard desktop components. Onboard Intel controller, H67, i5-2500, 16GB RAM.
 
performance problems are mustly due to:

OI:
- cheap (Realtek) nics
- bad disk (check system-statistic on disks)
- slow disks (like WD green), more RAM can help
- 4 k misalignment (vdevs of 4k or mixed disks with ashift=9)
- low RAM

ESXi
bad write values on NFS due to sync write
- disable sync on your NFS share or add a ZIL

Network
- bad cable or switch
- use a direct crossover cable

Windows
- cheap (Realtek) nic; use Intel or at least newest drivers
- copy tools like Teracopy, they slow down ZFS transfers
- test tools that are not suited for ZFS, try http://www.808.dk/?code-csharp-nas-performance

Mac
- lousy SMB Stack on Macs (only half the performance like on Windows. optionally use netatalk 3)

Overall:
Solaris ZFS use all available RAM for caching to deliver most reads from RAM so add RAM for read performance.
ZFS has a mechanism to improve (small) writes. All writes goes to RAM where 5s of writes are collected and then written
as one large and fast sequential write. So RAM is a ZFS performance factor.
 
_Gea said:
- I use a X9SRL-F as AV-Filer for OpenIndiana and have not had problems yet.
- Flash your HBA to LSI 9211 IT
- Do not care about PCI 4x in 8x slots - fast enough -
the slots are PCI-e 2.x and 3.x, fast enough even for next generation of HBA
Click to expand...

That's good news about the X9SRL-F. Seems like it might be a good idea to stick to Sandy Bridge cpu though. I guess I was staring too hard at the HBAs -- many thanks for the hints!

--Neil.

[ EDIT: On rechecking, I see the X9SRL-F only supports E5-16xx (& E5-2600). So Ivy Bridge isn't an option unless I'm missing something. ]
 
Last edited:
hominidae said:
...there are reports out with the X9SCM-F with IvyBridge CPUs, where passthrough of 3 M1015 would not work.
With SandyBridge CPUs all is fine. Also reported are effects/problems with other card types when using V2.0a (IvyBridge) BIOS, even when on SB-CPUs.
See: http://lime-technology.com/forum/ind...1887#msg201887
Click to expand...

Thanks for the heads-up, I was definitely considering a V2 CPU, but having read that thread may well now resist!

--Neil.
 
So if I read correctly, it is impossible to have 3 controllers with Ivy Bridge and the transfer speeds are slow(or at least parity checks). Now that sucks:) I guess I'll have to go with V1 as well...

Matej
 
...the X9SRL is SandyBridge and socket 2011, where the X9SCM is SandyBridge (upgradeable to IvyBridge with BIOS) on 1155 socket. So, all although being in the set of recommended X9xxx-F, the X9SRL is definitely a different kind with chipset 602.

The current reports with tze X9SCM might be temporary, as maybe a future BIOS fix might solve all that.
As the X9SRL + CPU is in a different league regarding price, I'd only jump on that if you can't wait and/or need the no. of PCIe slots on this.
 
Uff no, X9SRL is out of the question:) I was looking towards X9SCM+-F, but I guess I will have to go with Sandy Bridge if the troubles wont be resolved by the time... It might be hard to get the Sandy Bridge CPU though..

Matej
 
Anyone did test how much difference is there if you format a 4k drive with ashift=9 or ashift=12?

Matej
 
neiljt said:
Thanks for the heads-up, I was definitely considering a V2 CPU, but having read that thread may well now resist!

--Neil.
Click to expand...

im running a X9SCM-IIF with 3x m1015 with an e3-1230v2 and i have read that other people have gotten it to work with the latest bios.
have it running on vmware vsphere with passthrough mode with openindiana + nappit
 
bbzidane said:
im running a X9SCM-IIF with 3x m1015 with an e3-1230v2 and i have read that other people have gotten it to work with the latest bios.
have it running on vmware vsphere with passthrough mode with openindiana + nappit
Click to expand...

Thanks for the extra info -- good to hear!
 
Guys, was just wondering:
Do you use the OI Update Manager on a regular basis to update your OI NAS systems or do you leave it as is and just update Napp-IT..?
 
Hi guys ...

I'm having a problem managing automatic snapshots through the napp-it UI.

Jobs > snap > create autosnap job

I'm configuring everything as I'd like it to be and when I hit the submit button the page reloads, but it's not creating/displaying the job in the task list at the top of the page.

At first I thought this meant that it wasn't creating the task for some reason, however looking at my list of snapshots now, I can see that the jobs ARE being run, but they're still NOT being displayed at all on the autosnap page. This means that I've got no way of modifying/deleting these jobs through the napp-it UI.

When I thought that the jobs weren't being created properly, I tried configuring them multiple times, and now I've ended up with identical jobs that are creating a whole bunch of duplicate snapshots.

1. Does anyone know where napp-it actually saves the autosnap configuration when a job is created? In the short term, I want to just manually edit the jobs to remove the duplicate entries.

2. (This is aimed mainly at _Gea) Any ideas why this is happening in the first place, or perhaps more importantly, how I might fix it?

If it's useful at all, I'm running Solaris 11 (fully updated) and was originally using napp-it 0.8k. When I noticed the problem I tried updating napp-it to the latest version 0.8l3, but I'm still having the same problem.
 
We have a ZFS all-in-one on an HP ProLiant DL380 G6 server with 1 Xeon E5530 quad-core CPU, 32gb ECC memory, and one LSI 9211-8i SAS HBA (deployed to OI 151a6 via PCI passthrough). We were running ESXi 5.0 (free version) with the following VMS:
OpenIndiana (for virtual ZFS SAN)
Centos 6.3 (firewall, dns and email server, OpenVPN portal)
Oracle Linux 5.8 (based on RedHat EL 5.8) -- Oracle server
Windows 2008R2 -- Domain Controller

We have upgraded our VMWare license to the "Essentials Kit", which removes the 32gb memory limitation and can be deployed on up to three servers. We did this mostly because we are getting a new Dell server with 96gb of memory and would not be able to use the free edition. Since the Essentials kit allows up to three ESXi 5.1 servers, we decided to add memory to our existing server and upgrade it to 5.1.

The upgrade to ESXi 5.1 went smoothly. Then afterward when bringing up the VMs, there was a menu item to "Upgrade the Virtual Hardware" which we applied. After doing this the two linux-based VMs came up with FAILED networking. After a bunch of debugging, we discovered that the MAC address of the virtual NICs had changed (presumably from doing the "Upgrade Virtual Hardware" step). Since we were using static IP network configurations, I had to a) get the new NIC MAC addresses from vSphere Client, and b) edit the ifcfg-eth? files in /etc/sysconfig/network-scripts changing the HWADDR lines to reflect the new MAC addresses. If you have a RedHat EL 6 derived linux (such as our Centos 6.3) there is another required step in the udev configuration. The following link describes what to do:

Cloned-VMware-CentOS6-Server-and--quot-device-eth0-does-not-seem-to-be-present--delaying-initialization-Error

Hopefully this will save another ZFS all-in-one user some headaches!

--peter
 
You must log in or register to reply here.
Back
Top