OpenSolaris derived ZFS NAS/ SAN (OmniOS, OpenIndiana, Solaris and napp-it)

I do not really understand your problem at this point.
The function currently allows only integer data, nothing else.

You want comma values that is not possible in current release.
 
_Gea said:
I do not really understand
Click to expand...

It's OK. I got that by now.

Edit: Think hard about how your function handles 'Yes please, make me 1 volume of 10GB size' as an input, what it passes to zfs in that case and if any of that makes sense. Then you will know what input validation is if you seriously don't know at this point. Good night.
 
TCM2 said:
Fact is, napp-it is offered with a hefty price tag in certain licensing options, which invalidates your whole point about the good guy and "FOR FREE". I can't imagine the pro version to magically have better code. And of course you trust napp-it to handle your data correctly. If it trashes your data by calling zfs with some unvalidated input, it's not ZFS's fault, is it?
Click to expand...

I paid _Gea for a pro license, because the fancy monitoring applets are way easier on the eyes than iostat -xen 15. He did good work for a long assed time for free, and I figured the ease of use was worth the ~150 euros I paid him for his dev time.

The fact that there are odd interactions between various systems isn't too much of a surprise, nor is the fact that certain parts of Omni-OS have slightly different definitions of '2 TB'. If these issues get fixed in a newer version, that would be awesome, if not, I have documentation of my own with the workaround and commands used to un-fuck the iSCSI stuff. The most annoying part was figuring it out in the first place, after that the fix is two commands and a chkdsk on the isci target.
 
Eschertias said:
The fact that there are odd interactions between various systems isn't too much of a surprise, nor is the fact that certain parts of Omni-OS have slightly different definitions of '2 TB'.
Click to expand...

ZFS has exactly one definition of 2TB and that is 2199023255552 bytes. It is totally trivial to set the exact size when creating a volume and to query the exact size from a volume, down to the byte.

This is computer stuff, not guesswork or wishful thinking.
 
man zfs:

Code: 
         Numeric values can be specified as exact values, or in a human-read-
         able form with a suffix of B, K, M, G, T, P, E, Z (for bytes, kilo-
         bytes, megabytes, gigabytes, terabytes, petabytes, exabytes, or
         zettabytes, respectively).

2^x obviously.
 
TCM2 said:
man zfs:

Code: 
         Numeric values can be specified as exact values, or in a human-read-
         able form with a suffix of B, K, M, G, T, P, E, Z (for bytes, kilo-
         bytes, megabytes, gigabytes, terabytes, petabytes, exabytes, or
         zettabytes, respectively).

2^x obviously.
Click to expand...

Exact value is number of Bytes

2^x Bytes is KiB, MiB, GiB, TiB
10^x Bytes is KB, MB, GB and TB

I suppose I need to offer/display both options on input and output to avoid irritations
Sadly, there are many places where they are used incorrectly by many tools (Display GB when GiB is meant)


http://en.wikipedia.org/wiki/Kilobyte
The kilobyte is a multiple of the unit byte for digital information. Although the SI prefix kilo- means 1000, the term kilobyte and symbol KB have historically been used to refer to either 1024 (2^10) bytes or 1000 (10^3) bytes, dependent upon contex
 
Last edited:
_Gea said:
You need full access to modify permissions
(ex: disable guest, login as root)
Click to expand...

I disabled guest, connected using root account; i have full access to all the files and folders but i still can't change the share settings on the security tab from windows.

Second weird thing is that i can't find the "everyone" user anymore by searching for users/groups/... in the folders i can access their security tab.
 
I would reset the ACL of the filesystem using /usr/bin/chmod or napp-it ACL extension to a everyone@=modify, root=full or a basic root=full setting and then try again. Do not touch Unix permissions like 750 as they reset ACL inherheritance settings. Aclmode must be passthrough to modiy ACL in both cases.

The ACL everyone@ is not a user but refers to any known user - its a group that is similar to the Unix permissions for everyone.
 
Last edited:
Just accept GiB and GB and treat both as 2^x. Noone uses GB as 2^10 when creating a filesystem/volume, because 2^10 makes no sense whatsoever in storage[1], which is why this GiB nonsense is BS to begin with.

[1] as evidenced by the fact that the zfs man page knows nothing about GiB and Windows doesn't, either.
 
Last edited:
This is exactly why we do have this problem
They always use KB and sometimes they mean 1000 bytes othertimes 1024
depending on tools and situations.

example: If you buy a 1 TB disk its means 1000000000000 bytes mostly

The only way to overcome this situation is to use exact names for units instead of "it depends"
what means 1 kilo = 1000 and optionally offer both to be clear (beside the special case memory where 1KB=1KiB= 1024 bytes in nearly all cases).

http://en.wikipedia.org/wiki/Gigabyte
 
Last edited:
A little history lesson:

We never had any problem, except the marketing weasels of HDD manufacturers were selling us less capacity than advertised and being smartasses about it by claiming that technically, a GB is not a real GB and kicking off this whole GiB bullshit, despite the fact that 2^10 makes absolutely no sense in computer-related capacities. GB are not related to any other unit like 1 gram of water can be heated by 1 degrees Celsius using 1 calorie. There is no such connection since computers are 2^x based.

Why weren't RAM sticks only 32000000 bytes for 32MB? Because RAM manufacturers weren't that stupid to try to get away with such nonsense. HDD manufacturers got away and now we have this mess where existing units were redefined and new, completely useless units were introduced, amplifying this so-called "confusion" into unlimited proportions. It's literally like saying "Oh, I'm mildly confused so just let me fuck this up... Done. Wow, you're not using GiB. Now I'm _really_ confused! How dare you!"

I _never_ heard anyone who knew his stuff say "Gibibyte" in spoken language. Ever. It's just ridiculous. If you ever hear someone say that to you, you know you can ignore him for being an idiot. If I intentionally enter 1GB somewhere, I really mean 1073741824 bytes. Always.

I'm all for SI where it makes sense. Computing is not a field where it makes sense and apparently, ZFS itself, Windows, FreeBSD and numerous others feel the same. Look in your Windows drive properties, GB is 2^x based, even in Win8. Go shop for a RAM stick on Amazon, GB. Download a file in Firefox, MB.

All confusion about GB is self-induced by idiots intentionally misrepresenting GB for marketing or trolling reasons. HDDs are literally the _only_ place where GB is not 2^x based.

PS: Transmission speeds are a completely different matter since they're usually based on some frequency. Fast Ethernet operates at 100000000 Hz and not 104857600 Hz so it makes perfect sense that 100 Mbps are 100000000 bits per second. There is no confusion at all if you know your stuff.

Edit: As for the "problem" in napp-it, it's really trivial. First check for $number + ((K|M|G|T|P|...)i?)?B? suffix, then calculate it into bytes and use only bytes.
 
Last edited:
TCM2 said:
As for the "problem" in napp-it, it's really trivial. First check for $number + ((K|M|G|T|P|...)i?)?B? suffix, then calculate it into bytes and use only bytes.
Click to expand...

That would probably be the for the best. ZFS list -p gives raw byte values, so the volume creation stuff, silly regex issue aside, works fine.

I think the bug may be someplace in the COMSTAR volume LU detection and size conversion code. It autodetects the volumes just fine, but doesn't detect the size right, or it does, but thinks it needs the GB->GiB conversion and ends up always making it 11% larger than the volume hosting it.
 
If someone wants to try on napp-it Pro:

0.9unstable (0.9f2_dev) displays and allows to set dec (GB)
and bin units (GiB) as well as decimal inputs.
 
Has anybody out there experience with PCIe Extension Cables?
I have to use one cause of my case U-NAS NSC-800 to get the IBM 1015/LSI 9211-8i IT working.
omnios r101015u states the following error that isn't looking good.
I've already tried four different cables but the result is always the same.
Without PCIe Extension Cable the everything is fine.
omnios seems to be very picky when the communication over the bus isn't perfect.
With the same hardware and ubuntu installed I couldn't see a similar message.
I'm not an expert but ubuntu /var/log/syslog didn't mention anything.
Any ideas?

Code: 
--------------- ------------------------------------  -------------- ---------
TIME            EVENT-ID                              MSG-ID         SEVERITY
--------------- ------------------------------------  -------------- ---------
Sep 20 14:06:43 abb38e47-cd60-6638-bd03-bbd5cf1bbd51  PCIEX-8000-KP  Major     

Host        : sefs
Platform    : A1SAi	Chassis_id  : 123456789
Product_sn  : 

Fault class : fault.io.pciex.device-interr-corr max 50%
              fault.io.pciex.bus-linkerr-corr 25%
Affects     : dev:////pci@0,0/pci8086,1f12/pci1000,3020@0
              dev:////pci@0,0/pci8086,1f12@3
                  faulted and taken out of service
FRU         : "MB" (hc://:product-id=A1SAi:server-id=sefs:chassis-id=123456789/motherboard=0)
                  faulty

Description : Too many recovered bus errors have been detected, which indicates
              a problem with the specified bus or with the specified
              transmitting device. This may degrade into an unrecoverable
              fault.
              Refer to http://illumos.org/msg/PCIEX-8000-KP for more
              information.

Response    : One or more device instances may be disabled

Impact      : Loss of services provided by the device instances associated with
              this fault

Action      : If a plug-in card is involved check for badly-seated cards or
              bent pins. Otherwise schedule a repair procedure to replace the
              affected device.  Use fmadm faulty to identify the device or
              contact Sun for support.
 
So I'm getting ready to start my napp-it build - I've been lurking in this thread for a very long time. Still have not had the chance to read all 102 pages but I'm trying.

I was hoping someone could look over my setup and give any comments -

What I have

GA-p55-ud4p with i5 quad core, VT-D enabled, 8GB DDR3 1600mhz
LSI 9207-8i - passthrough enabled
4x Seagate 7200rpm 4TB drives
2x USB 3.0 60gb sticks
250GB Samsung SSD - not sure I need to use this if esxi is on the USB sticks?

What I want to run is 3x VMs
1. CentOS VM - I have a 1u in a datacenter that I am migrating back that does pentesting/webhosting - this is for my infosec business so the whole home use of the OS might come into play here
2. SageTV 32bit windows 2008 server - heart of our PVR system in the house - it needs the most storage
3. Windows 2012 - for backups - mostly documents folders and the like

Edit: I'm assuming since there is a VM with Omni OS on the napp-it site ready to download that it is a safe bet? I'm up to trying another OS but haven't found any recommendations.

For raid I was going to use Z2 since I have 4 disks. Should I use the SSD for anything?

Any help is appreciated and Gea it is totally kick ass that you have put this all together!
 
Last edited:
enigmah said:
Has anybody out there experience with PCIe Extension Cables?
I have to use one cause of my case U-NAS NSC-800 to get the IBM 1015/LSI 9211-8i IT working.
omnios r101015u states the following error that isn't looking good.
I've already tried four different cables but the result is always the same.
Without PCIe Extension Cable the everything is fine.
omnios seems to be very picky when the communication over the bus isn't perfect.
With the same hardware and ubuntu installed I couldn't see a similar message.
I'm not an expert but ubuntu /var/log/syslog didn't mention anything.
Any ideas?
Click to expand...

I'd step back and ask myself why the hell I would want to introduce the complexity and error proneness of a PCIe expander in such a crucial system as a NAS.

I don't really know what you want to hear. How to get rid of the error? It's possibly there for a reason. How to get rid of the cause of the error? I think you have nailed the cause pretty well and 4 different cables giving the same error highly suggests that there really isn't any workaround. It could in part be the controller's fault as well when plugged into an expander.

The fact that Linux shows no error would rather worry me than provide confidence in Linux.

Sorry to say that, but I'd re-evaluate the choice of hardware. That case in particular seems like too big of a compromise if it necessitates the use of a PCIe expander. Switch the case or use a board that doesn't need the M1015.
 
Eschertias said:
That would probably be the for the best. ZFS list -p gives raw byte values, so the volume creation stuff, silly regex issue aside, works fine.

I think the bug may be someplace in the COMSTAR volume LU detection and size conversion code. It autodetects the volumes just fine, but doesn't detect the size right, or it does, but thinks it needs the GB->GiB conversion and ends up always making it 11% larger than the volume hosting it.
Click to expand...

How do you mean always makes it 11% larger?
Sorry, not sure where this is coming from.

you make the lun EXACTLY the size requested. Comstar will use that size exactly (maybe remove 64k for metadata info for the lun serial number and other params).

If you use a zvol, you could be adding a lot of metadata, if you make the blocksize small, or even leave it at the default of 8k.
 
I have been updating our [virtual] linux servers (mix of Centos 6.5, Oracle Linux 5.11, Oracle Linux 6.5) to remediate the Shellshock bash bug. We have two VMWare Esxi 5.1 hosts, each with an OpenIndiana VM used as a ZFS SAN for an all-in-one. As far as I can tell there is no bash update for OI at this time. Nor Omnios. What are all you all-in-one users doing about this?
 
So I'm a bit stuck - I'm having issues with napp-it recognizing my drives.

Motherboard: Supermicro X10SL7-F w/ LSI 2308 successfully flashed to IT mode - All drives show up in the LSI bios/manager and it shows IT mode.

Main VM Drive 250gig SSD (working fine)
Storage - 4x Seagate 4tb's off of the 2308


I have the VM installed to an SSD off of the built in Intel adapter on the motherboard. Napp-it installed successfully with the fully built VM. Passthrough enabled. On the VM it defaulted to LSI Parallel - if I change it to LSI SAS the Napp-it vm reboots constantly so I'm assuming LSI Parallel is right.

hakOMK7.png

Jqbma7o.png

V7qMDUV.png

I4VbLDr.png

0B1SNXP.png


Any ideas what I am doing wrong or are there some tools that I can get some more information out of the drives? I found another thread about the 2308 and said person had to update a driver - I tried that and it would not apply.

I have a 9207-8i as well but was trying not to use it since I have the 2308 on the motherboard and uses the same chipset.

If I remove the "passthrough" the drives show back up in the datastore

cQzhxCE.png
 
TCM2 said:
You boot from the virtual disk, right? It's attached to that controller, so no.
Click to expand...

Edit:

I boot from the Virtual Disk which is an SSD on a different controller - the motherboard has the standard intel controllers as well as the LSI 2308

The VM had the LSI as a SCSI parallel device (top red box) before it worked

After I added it as a PCI device (bottom red box) it works now

Since it doesn't seem to have any issues I guess I will leave the SCSI device as is

ypfr30U.png


Thanks for your help
 
I think there's still some confusion. The "SCSI controller 0" has nothing to do with your physical LSI controller. It's a purely virtual device which hosts the "Hard disk 1" virtual disk. You can present that virtual controller as different devices to the VM, depending on the OS's driver status.

If you remove it, your "Hard disk 1" goes away as well and the VM has nothing to boot. Edit: Rather, you can't remove it, because a virtual disk depends on it. It automatically goes away if you remove all virtual disks.
 
TCM2 said:
I think there's still some confusion. The "SCSI controller 0" has nothing to do with your physical LSI controller. It's a purely virtual device which hosts the "Hard disk 1" virtual disk. You can present that virtual controller as different devices to the VM, depending on the OS's driver status.

If you remove it, your "Hard disk 1" goes away as well and the VM has nothing to boot. Edit: Rather, you can't remove it, because a virtual disk depends on it. It automatically goes away if you remove all virtual disks.
Click to expand...

Thanks, I think I got it!
 
So everything is working good now - except...

I have 10tb of storage on NFS on the Napp-it. If I create VMs under 2tb there are no issues.

I want a 2012 with over 2TB storage. I did a 2012 install - even though it registers 8tb in disk management, only 2tb is usable.

I've been scouring the net but have not found any easy way to do this. This server is my SageTV media server so I need the space. Vmware says that it supports up to 62tb but maybe not over NFS?
 
i'm planning to upgrade my pool.
from:
6x 2tb raidz1
to
8x 4tb raidz2

i've always used hitachi disks and had good results. have to make a new choice now.
what about the :

WD Red SATA 6 Gb/s WD40EFRX, 4TB
150euro each.

please give me your opinion. its for home use. esxi with some servers. 24/7 on. mostly for storing videofiles.
 
I want to move from a baremetal install of OmniOS+Napp-it to the VMWare all-in-one. I have both running concurrently now, but of course all of the data resides on the baremetal. If I physically move the disks (perform a zfs export beforehand) then will VMWare require me to format them to VMFS before they are available for use? What's the accepted procedure for moving from baremetal to AIO ? Disks are in a mirror (2x2TB)

If ESXi requires me to format the disks, I suppose I could move just one, since they are a mirror, format it, then transfer the pool over?
 
nl-darklord said:
i've always used hitachi disks and had good results. have to make a new choice now.
what about the :

WD Red SATA 6 Gb/s WD40EFRX, 4TB
150euro each.

please give me your opinion. its for home use. esxi with some servers. 24/7 on. mostly for storing videofiles.
Click to expand...

Why not HGST then. HGST Deskstar NAS or (if you can find it, as the availability is poor) Deskstar Coolspin (5900 RPM).
 
Dark Shade said:
I want to move from a baremetal install of OmniOS+Napp-it to the VMWare all-in-one. I have both running concurrently now, but of course all of the data resides on the baremetal. If I physically move the disks (perform a zfs export beforehand) then will VMWare require me to format them to VMFS before they are available for use? What's the accepted procedure for moving from baremetal to AIO ? Disks are in a mirror (2x2TB)

If ESXi requires me to format the disks, I suppose I could move just one, since they are a mirror, format it, then transfer the pool over?
Click to expand...

Pass the controller that has the disks through to the VM and import the pool. Do not let ESXi touch the disks in any way.
 
nl-darklord said:
i'm planning to upgrade my pool.
from:
6x 2tb raidz1
to
8x 4tb raidz2

i've always used hitachi disks and had good results. have to make a new choice now.
what about the :

WD Red SATA 6 Gb/s WD40EFRX, 4TB
150euro each.

please give me your opinion. its for home use. esxi with some servers. 24/7 on. mostly for storing videofiles.
Click to expand...

That disks are not bad for that price, we are using them in many ZFS pools for about year. Some of them were replaced because of failure, and their performence is worse, than WD RE4 (great disks by the way). But now, I would recommend WD RED Pro (WD4001FFSX). They are quite new, but their specs are near to WD RE4. We are using them in one 10x 4TB ZFS pool and seem to be good. But it's short time for conclusion.
 
spazoid said:
Pass the controller that has the disks through to the VM and import the pool. Do not let ESXi touch the disks in any way.
Click to expand...

Yeah I was going to do that but my server doesn't do VT-d, so I had to split the two drives, format one of them in the ESXi instance and manually copy over the files from the original ZFS server to the new one. Thankfully I have less than 15 VMs I had to do this for. I'll be re-adding the disk from the original server to the new one to recreate the mirror.
 
You must log in or register to reply here.
Back
Top