![[H]ard|Forum](/styles/hardforum/xenforo/logo_dark.png){kind=logo}
OpenSolaris derived ZFS NAS/ SAN (OmniOS, OpenIndiana, Solaris and napp-it)
- Thread starter _Gea
- Start date
nostradamus99
Weaksauce
- Joined
- Jun 28, 2012
- Messages
- 88
You can also boot your system using ZFSGuru to create a 4K ready pool in a safe way after that you can export the pool and use Napp-IT to import it
This is an amazing product. This should be the poster-child for open-source projects. I'd been banging my head against the wall trying to get decent NFS performance between a linux client and a nas4free server, and after finding this thread I knew I found the right solution.
Fired it up. Added an NFS client and I was saturating my gig connection with no other tweaks.
I'm only having one problem. My client can't set permissions within the ZFS share. I've seen similar ports reported in this thread, but I'm not familiar enough with the GUI to folow the provided instructions. Here's what I see under home >> ZFS Folder
.
And under home >> Extensions >> acl-settings >> ACL on folders
And this is what I see on the Debian client
Here's the mount info from the client
The system is running on OpenIndiana text-version and napp-it v 08133 nightly Sep.20.2012
One other question. We're considering installing a backup nas in an active/passive configuration with HA. And then a second set of mirrored NAS's in our backup data center. So the same data needs to be on a total of 4 NAS's. What's the best way to accomplish this? Does this set require a total of 3 of the HA licenses?
Fired it up. Added an NFS client and I was saturating my gig connection with no other tweaks.
I'm only having one problem. My client can't set permissions within the ZFS share. I've seen similar ports reported in this thread, but I'm not familiar enough with the GUI to folow the provided instructions. Here's what I see under home >> ZFS Folder
.
And under home >> Extensions >> acl-settings >> ACL on folders
And this is what I see on the Debian client
Here's the mount info from the client
The system is running on OpenIndiana text-version and napp-it v 08133 nightly Sep.20.2012
One other question. We're considering installing a backup nas in an active/passive configuration with HA. And then a second set of mirrored NAS's in our backup data center. So the same data needs to be on a total of 4 NAS's. What's the best way to accomplish this? Does this set require a total of 3 of the HA licenses?
1.
-NFS3 share access is based on client IP's only. There is no user authentication.
-Local file access and CIFS are based on user authentication and user/group-ACL.
These two approaches are incompatible.
Best ways to have NFS3 working are:
- your client has root permission (root=option on sharing)
more: google for " share nfs options solaris unix"
and/or
- you have set an ACL like everyone@=modify with inheritance = on for your shared folder
or
- use CIFS (best interoperability of all protocols)
2.
you should ask highavailabilty.com
but i suppose, you need 4 licences
-NFS3 share access is based on client IP's only. There is no user authentication.
-Local file access and CIFS are based on user authentication and user/group-ACL.
These two approaches are incompatible.
Best ways to have NFS3 working are:
- your client has root permission (root=option on sharing)
more: google for " share nfs options solaris unix"
and/or
- you have set an ACL like everyone@=modify with inheritance = on for your shared folder
or
- use CIFS (best interoperability of all protocols)
2.
you should ask highavailabilty.com
but i suppose, you need 4 licences
turquoisewords
n00b
- Joined
- Jan 17, 2012
- Messages
- 42
We just got a new Dell r720xd server that came with a built-in Dell Perc h310 raid controller [based on LSI SAS 2008 controller] (looks like a daughter card on the motherboard as opposed to a PCIe add-on card). We also purchased an LSI 9211-8i (which we have used with great success on an older HP server). The Perc h310 has SFF-8087 cabling to a SAS backplane with 12 hot-swap SAS 3.5in disks (actually these are near-line SAS drives). We have two Corsair Force 60gb SSD drives for boot drives. Since the cables that came with our LSI 9211-8i are SFF-8087 at the controller end with 4 fan-out SAS/Sata connectors at the drive end we decided to use the 9211 in standard LSI IR mode [hardware raid] and make a raid 1 mirror of the two SSDs as a boot drive for ESXi 5.1. Booted ESXi 5.1 and configured an OpenIndiana VM with the original Dell Perc h310 passed through (after setting each drive as what Dell calls "non-raid" which is Dell speak for JBOD).
Our problem: after OI boots and comes up with the Gnome desktop, an error comes that there is unrecognized hardware. The Dell Perc h310 shows up as:
LSI Logic / Symbios Logic MegaRAID SAS 2008 [Falcon]
and for driver OI has "UNK". Before bringing up OI I tried to flash the Dell to the LSI 9211 IT mode firmware but the only controller that showed up was the actual 9211 add-in card -- not the Perc h310.
At this point our options appear to be either:
1. Find a suitable Solaris driver for the Dell Perc h310 which so far has eluded us and a Google search.
2. Buy two SFF-8087 multi-lane cables and connect the SAS backplane to the 9211-8i and then somehow use the fan-out cables from the Dell Perc h310 and set up a raid 1 volume of the SSDs on that. Then use PCI passthrough of the 9211-8i instead of the Dell Perc.
3. Find different LSI firmware that can flash the Dell Perc h310 to something that OI can recognize.
Any and all help gratefully requested!!
--peter
Our problem: after OI boots and comes up with the Gnome desktop, an error comes that there is unrecognized hardware. The Dell Perc h310 shows up as:
LSI Logic / Symbios Logic MegaRAID SAS 2008 [Falcon]
and for driver OI has "UNK". Before bringing up OI I tried to flash the Dell to the LSI 9211 IT mode firmware but the only controller that showed up was the actual 9211 add-in card -- not the Perc h310.
At this point our options appear to be either:
1. Find a suitable Solaris driver for the Dell Perc h310 which so far has eluded us and a Google search.
2. Buy two SFF-8087 multi-lane cables and connect the SAS backplane to the 9211-8i and then somehow use the fan-out cables from the Dell Perc h310 and set up a raid 1 volume of the SSDs on that. Then use PCI passthrough of the 9211-8i instead of the Dell Perc.
3. Find different LSI firmware that can flash the Dell Perc h310 to something that OI can recognize.
Any and all help gratefully requested!!
--peter
Is there an easy way to determine NIC saturation in OI? Right now I am running on the 2 gigabit ports on my Rackable setup and I am thinking of adding some SSD storage somewhere for our databases. I just wanted to make sure that the current NICs can handle the load, but I need to have some metrics to go by.
I love the setup at http://www.zfsbuild.com/ and I think that's a nice long term goal for our storage. OI and Napp-It have been working very well so far for all of our backup storage and this is on some very low end hardware. Once I can prove some performance numbers and reliability to management then I think a big SuperMicro based storage server will be an easy sell.
EDIT:
nicstat 1.92 worked. I pulled the latest version from SourceForge and while the shell script didn't know about OI, I was able to run it with:
./.nicstat.Solaris_11_i386 1
After that I was able to see nge0 and nge1. Other versions of nicstat just saw lo0 and nothing else. This will help a lot in working through our current networking needs.
I love the setup at http://www.zfsbuild.com/ and I think that's a nice long term goal for our storage. OI and Napp-It have been working very well so far for all of our backup storage and this is on some very low end hardware. Once I can prove some performance numbers and reliability to management then I think a big SuperMicro based storage server will be an easy sell.
EDIT:
nicstat 1.92 worked. I pulled the latest version from SourceForge and while the shell script didn't know about OI, I was able to run it with:
./.nicstat.Solaris_11_i386 1
After that I was able to see nge0 and nge1. Other versions of nicstat just saw lo0 and nothing else. This will help a lot in working through our current networking needs.
Last edited:
maybee this one for an ITX server?
http://www.intel.com/content/www/us/en/motherboards/server-motherboards/server-board-s1200kp.html
I have not tried but it is based on a intel server chipset with ECC, vt-d and Xeon Support (ex. 20W low power ones, but I would always prefer uATX as the smallest size due to more PCI slots and more possible RAM).
turquoisewords
n00b
- Joined
- Jan 17, 2012
- Messages
- 42
ESXi seems to have no problem with the Dell Perc H310. -- it is OpenIndiana that has the problem. The issue with #2 is that the embedded Dell Perc controller seems to use a custom SAS connector at the controller end that looks like two SFF-8087s joined together. I haven't removed it yet and will have to investigate. Right now I am flashing the H310 to LSI MegaRAID firmware 20.10.1-0061. My first attempt to do this failed because the Dell firmware had version 20.11.0-002 and the flash program (MegaCLI) gave an error that the firmware version was older than what was on the controller. I did discover that there is a "-NoVerChk" option to the flash command and I will try that shortly.
This has been quite frustrating!
--peter
thanlks for the info, I will include in next napp-it 0.9
Hey there!
I have a share and I want to assign a user with certain permissions to it. I log to the share as a root and try to add the user, but I get an error saying: Unable to lookup user names for display.
I can add Power users and Administrators group, but can't add SMB user i created in nappit... What am I doing wrong?
lp, Matej
I have a share and I want to assign a user with certain permissions to it. I log to the share as a root and try to add the user, but I get an error saying: Unable to lookup user names for display.
I can add Power users and Administrators group, but can't add SMB user i created in nappit... What am I doing wrong?
lp, Matej
turquoisewords
n00b
- Joined
- Jan 17, 2012
- Messages
- 42
We are up and running. Napp-it sees the 8 2TB drives on the SAS backplane. If anyone else is trying this, here are the steps taken:
1. Install and configure OpenIndiana as a VM under ESXi.
2. Download the following Solaris/OI driver:
http://www.lsi.com/downloads/Public/MegaRAID Common Files/3.03_2008_Solaris_Driver.zip
3. Follow instructions in the README to install the driver (imr_sas) in your OI host.
4. Power down your OI VM.
5. Put ESXi host into maintenance mode and power down the Dell server (ours is an r720xd, but probably any Dell server with Perc H310 will work the same).
6. Make a bootable DOS pen drive.
7. Flash the Dell Perc H310 to LSI MegaRAID firmware 20.10.1-0107 from:
http://www.lsi.com/channel/products/storagecomponents/Pages/MegaRAIDSAS9240-8i.aspx#Driver (click on the Firmware choice).
You need LSI's MegaCLI DOS exe for this (put on pen drive).
Command syntax: MegaCLI -adpfwflash -f imr_fw.rom -NoVerChk -a0
Without the "-noVerChk" you will get an error that the firmware on the device is newer than the one you are trying to flash.
8. Power on the server and boot ESXi.
9. Set ESXi to use PCI passthrough of the controller.
10. Add controller to OI VM configuration.
11. Bring up OI VM. It should recognize the controller and use the imr_sas driver.
12. Install napp-it and proceed to Valhalla :=)
--peter
This a problem with some Windows editions - mostly home editions.
You can use the napp-it ACL extension to add users (adding local users is free in this extension)
I keep having errors with my server lately. I've known that when my server restarts a few disks pop up with too many errors and it has always been able to clear the errors fine. Now it just says they are unavailable but the disks are all there and being read by the OS. is there something i can try at the command line to get it back up, i reallly need a file off of it like ASAP then after that i can work on getting some shorter SAS cables like i was recommended. I'm gonna try reinstalling the OS and see if it wasn't just a bad OS install in the first place, but what else could it be?
EDIT: where are some logs to look at even?
Output of the pool status
zpool status
pool: Documents2
state: UNAVAIL
status: One or more devices are faulted in response to persistent errors. There are insufficient replicas for the pool to
continue functioning.
action: Destroy and re-create the pool from a backup source. Manually marking the device
repaired using 'zpool clear' may allow some data to be recovered.
scan: none requested
config:
NAME STATE READ WRITE CKSUM CAP Product
Documents2 UNAVAIL 0 0 0 insufficient replicas
raidz1-0 UNAVAIL 0 0 0 insufficient replicas
c0t50014EE2AE7F266Ad0 ONLINE 0 0 0 1000.20 GB WDC WD1001FALS-0
c0t50014EE056931134d0 ONLINE 0 0 0 1000.20 GB WDC WD1001FALS-0
c0t50014EE0ABE88295d0 ONLINE 0 0 0 1000.20 GB WDC WD1001FALS-0
c0t50014EE0ABE88280d0 ONLINE 0 0 0 1000.20 GB WDC WD1001FALS-0
c0t50014EE2AE7F2D18d0 FAULTED 0 0 0 too many errors 1000.20 GB WDC WD1001FALS-0
c0t50014EE0ABE0C365d0 FAULTED 0 0 0 too many errors 1000.20 GB WDC WD1001FALS-0
c0t50014EE2AD2D485Fd0 ONLINE 0 0 0 1000.20 GB WDC WD10EADS-00L
c0t50014EE001726B37d0 ONLINE 0 0 0 1000.20 GB WDC WD1001FALS-0
c0t50014EE25898D604d0 ONLINE 0 0 0 1000.20 GB WDC WD10EADS-00M
c0t50014EE2AD2D4A34d0 ONLINE 0 0 0 1000.20 GB WDC WD10EADS-00L
raidz1-1 DEGRADED 0 0 0
c0t5000CCA221D562CAd0 ONLINE 0 0 0 2.00 TB Hitachi HDS72202
c0t5000CCA369C3B912d0 ONLINE 0 0 0 2.00 TB Hitachi HDS72302
c0t5000CCA36ACC7314d0 ONLINE 0 0 0 2.00 TB Hitachi HDS72302
c0t5000CCA221D55200d0 FAULTED 0 0 0 too many errors 2.00 TB Hitachi HDS72202
c0t5000CCA369CD2708d0 ONLINE 0 0 0 2.00 TB Hitachi HDS5C302
c0t5000CCA369CD251Ad0 ONLINE 0 0 0 2.00 TB Hitachi HDS5C302
EDIT: where are some logs to look at even?
Output of the pool status
zpool status
pool: Documents2
state: UNAVAIL
status: One or more devices are faulted in response to persistent errors. There are insufficient replicas for the pool to
continue functioning.
action: Destroy and re-create the pool from a backup source. Manually marking the device
repaired using 'zpool clear' may allow some data to be recovered.
scan: none requested
config:
NAME STATE READ WRITE CKSUM CAP Product
Documents2 UNAVAIL 0 0 0 insufficient replicas
raidz1-0 UNAVAIL 0 0 0 insufficient replicas
c0t50014EE2AE7F266Ad0 ONLINE 0 0 0 1000.20 GB WDC WD1001FALS-0
c0t50014EE056931134d0 ONLINE 0 0 0 1000.20 GB WDC WD1001FALS-0
c0t50014EE0ABE88295d0 ONLINE 0 0 0 1000.20 GB WDC WD1001FALS-0
c0t50014EE0ABE88280d0 ONLINE 0 0 0 1000.20 GB WDC WD1001FALS-0
c0t50014EE2AE7F2D18d0 FAULTED 0 0 0 too many errors 1000.20 GB WDC WD1001FALS-0
c0t50014EE0ABE0C365d0 FAULTED 0 0 0 too many errors 1000.20 GB WDC WD1001FALS-0
c0t50014EE2AD2D485Fd0 ONLINE 0 0 0 1000.20 GB WDC WD10EADS-00L
c0t50014EE001726B37d0 ONLINE 0 0 0 1000.20 GB WDC WD1001FALS-0
c0t50014EE25898D604d0 ONLINE 0 0 0 1000.20 GB WDC WD10EADS-00M
c0t50014EE2AD2D4A34d0 ONLINE 0 0 0 1000.20 GB WDC WD10EADS-00L
raidz1-1 DEGRADED 0 0 0
c0t5000CCA221D562CAd0 ONLINE 0 0 0 2.00 TB Hitachi HDS72202
c0t5000CCA369C3B912d0 ONLINE 0 0 0 2.00 TB Hitachi HDS72302
c0t5000CCA36ACC7314d0 ONLINE 0 0 0 2.00 TB Hitachi HDS72302
c0t5000CCA221D55200d0 FAULTED 0 0 0 too many errors 2.00 TB Hitachi HDS72202
c0t5000CCA369CD2708d0 ONLINE 0 0 0 2.00 TB Hitachi HDS5C302
c0t5000CCA369CD251Ad0 ONLINE 0 0 0 2.00 TB Hitachi HDS5C302
Last edited:
turquoisewords
n00b
- Joined
- Jan 17, 2012
- Messages
- 42
I am afraid I was a bit premature about everything working. Although napp-it could see the 8 2TB disks, when I tried to create a pool, it failed. The disks were not accessible. At this point we decided to order another LSI 9211 and disconnect the Dell Perc H310 from the SAS backplane. Hope I didn't mislead anyone.
--peter
mmmmmdonuts
Gawd
- Joined
- Oct 7, 2009
- Messages
- 821
I have the desktop edition of OI151a5 on ESXi 5.1 and have been trying to log into the desktop after I changed my PW of the user on napp-it. It allows me to log in, but it asks me to redirect my shares and regardless if I say yes or no, it fails and the desktop does not open. Anyone have any ideas short of reinstalling the operating system? I can access napp-it with zero issue. Thanks.
The napp-it users admin and operator are napp-it users - independend from the base OS
If you have have problems with Solaris users, you can handle them with the root account.
If you cannot use the root account for whatever reason, you must reinstall OS, install napp-it and importt your pool
and recreate then regular users.
mmmmmdonuts
Gawd
- Joined
- Oct 7, 2009
- Messages
- 821
Okay I never set them up as a napp-it user. They were setup on the change user screen when you access the UNIX/CIFS users, where I can assign ACL and permissions. Does this change anything?
as I know,
you should flash with Dell host 6G HBA firmware, this is the easy way without messing flashing with LSI firmware since Dell Sas card is kind of picky..(not like M1015)
I'm looking at adding an SSD for ZIL shortly and I am seeing the Intel 313 SLC drives recommended highly, but I am also looking at the Intel 520 drives for L2ARC. Due to the size of the Intel 520 drives, could I partition those into ZIL and L2ARC since the 520 drives have pretty good IOPS? I know the MLC drives will wear faster than the SLC drives, but in a year I am assuming SSDs will keep coming down in the $/GB area AND get faster. I don't mind eating the 4 slots for the drives, but if I could do it in 2 that would be even better.
turquoisewords
n00b
- Joined
- Jan 17, 2012
- Messages
- 42
Could you please give me more info about "Dell host 6G HBA firmware"? Link would be great.
We ordered a 2nd LSI 9211 and some SFF-8087 right-angle cables to connect to the Dell r720xd SAS backplane. (Of course we first disconnected the Dell custom one-into-two SAS cable that was connected to the Perc H310). The standard right-angle connector SFF-8087 does not fit into the backplane connector because it is oriented in the wrong direction! The only way it could fit would be to insert it backwards (with the metal clip facing down instead of up). I doubt that SFF-8087 is symmetrical so it can only be oriented with the flat part of the connector down and the metal clip part up. At this point it appears our only option is to go with straight SFF-8087 to SFF-8087 cables. Using that we will have to bend the cable down in urder to get the cover back on. I am worried that the strain on the cable might not be a good idea.
In total frustration,
--peter
Mates,
Im currently using OI+Napp-it installed on a external usb disk for testing and its working great. Im looking forward to put this server on production so i need to get rid of this usb disk asap.
Which option would be the best: a cheap 500gb hdd for $100 or a 64gb ssd for $200.
Yes, im outside-us so the prices arent too attractive
If i install OI on this Vertex4, the spare can be partitioned to be used as cache? Is it worth it?
The server is no big deal. A Dell T420 with 8Gb ECC memory and a 3x2Tb Raidz1 over a M1015.
Most of the files are office documents (10+ users) with some sporadic write of 2Gb+ media files.
Thanks in advance
Im currently using OI+Napp-it installed on a external usb disk for testing and its working great. Im looking forward to put this server on production so i need to get rid of this usb disk asap.
Which option would be the best: a cheap 500gb hdd for $100 or a 64gb ssd for $200.
Yes, im outside-us so the prices arent too attractive
If i install OI on this Vertex4, the spare can be partitioned to be used as cache? Is it worth it?
The server is no big deal. A Dell T420 with 8Gb ECC memory and a 3x2Tb Raidz1 over a M1015.
Most of the files are office documents (10+ users) with some sporadic write of 2Gb+ media files.
Thanks in advance
Mates,
Im currently using OI+Napp-it installed on a external usb disk for testing and its working great. Im looking forward to put this server on production so i need to get rid of this usb disk asap.
Which option would be the best: a cheap 500gb hdd for $100 or a 64gb ssd for $200.
Yes, im outside-us so the prices arent too attractive
If i install OI on this Vertex4, the spare can be partitioned to be used as cache? Is it worth it?
The server is no big deal. A Dell T420 with 8Gb ECC memory and a 3x2Tb Raidz1 over a M1015.
Most of the files are office documents (10+ users) with some sporadic write of 2Gb+ media files.
Thanks in advance
You can use any Sata disk greater 16GB as boot disk - performance is not critical.
I would add some RAM. This is far more effective than a SSD Cache.
turquoisewords
n00b
- Joined
- Jan 17, 2012
- Messages
- 42
We did purchase another LSI 9211-8i and had a dog of a time cabling it to the Dell r720xd SAS backplane. Unfortunately after getting OI powered on with the backplane cabled in, we found the same problem -- the disks were recognized but could not be accessed. The errors in /var/adm/messages were like this:
Doing some google research I found a fix to this problem in Illuminos Bug #2091 with a workaround. Like the vshift fix, I had to add a stanza to /kernel/drv/sd.conf:
Most of what I read said that after making changes to sd.conf, you need to run the following command to restart the sd driver:
# update_drv -vf sd
Well, unfortunately that did not make the drives power on. Rebooting OI, however, worked.
We are now getting really FANTASTIC disk I/O performance as measured via the napp-it dd benchmark:
562.99 Mb/s Write
864.80 Mb/s Read
Needless to say we are now very happy. We had decided to abandon OI/Solaris and use Oracle Linux 6.3 with the UEK2 which has official support for BTRFS. But now we remain in the OI/napp-it camp.
--peter
Currently, you have the best overall ZFS experience when you are using a modern SuperMicro Server based on Intel servermainboards, original LSI controllers in IT mode (or LSI reflashed like IBM 1015 flashed to LSI 9211) and Intel Nics in a whitebox config.
Dell is quite near to a optimal ZFS configuration. They use Intel mainboards, Intel Nics and LSI based controllers. Main problem is that they are too Microsoft focussed. This results in LSI controllers with hardware-raid firmware only (generic LSI firmware for IT/HBA mode often not available) and "Dell certified disks" with a special firmware that do not work with ZFS softwareRaid /Solaris out of the box without modifying sd.conf (in contrast to the same disks form original manufacturers like Seagate). It seems that Dell is now aware of the problem and we may hope for a supported Dell config. You should ask your Dell reps for a better ZFS/Solaris support as well.
Last edited:
it is coming. the 60 drive JBOD of theirs is already confirmed working, and working well. i know of a pretty large installation that just occurred with a boat load of the 12 drive dell JBODs.
beware of the intel i350 nics though. there is currently a strange bug that causes the NICs to fall asleep. not just a dell problem either.
Yes I know of efforts from Dell system houses evaluating large ZFS Dell installations. I do hope for a Dell supported config as well.
But my next ZFS config (up to 200 TB raw capacity, low cost) will be based on a 9 HE Chenbro case with 48 hotswap slots, 3 x 16 port LSI HBA and a 7 x PCI-e SuperMicro mainboard + enough 4 TB SATA disks - no comparable Dell offer currently available -not even for the double price-
(expected costs: about 20 000 Euro/ 26 000 USD per about 200 TB raw capacity)
Last edited:
have you checked out these yet?
http://www.quantaqct.com/en/01_product/02_detail.php?mid=29&sid=143&id=144&qs=96
list price from my VAR was ... $5600usd? something in the $5500 range. they've made that same chassis for awhile but they have a new one with more expanders within the chassis and LSI FRUs. the older model only had one or two expanders with PMC sierra FRUs. the single expander had issues for the drives that were electronically furthest from the source ports. this new one doesn't.
i'm still hoping LSI comes out with a 16 port x16 pci-e 3.0 card. they had a pci-e 2 x16 with 16 ports but it was hard to get.
http://www.quantaqct.com/en/01_product/02_detail.php?mid=29&sid=143&id=144&qs=96
list price from my VAR was ... $5600usd? something in the $5500 range. they've made that same chassis for awhile but they have a new one with more expanders within the chassis and LSI FRUs. the older model only had one or two expanders with PMC sierra FRUs. the single expander had issues for the drives that were electronically furthest from the source ports. this new one doesn't.
i'm still hoping LSI comes out with a 16 port x16 pci-e 3.0 card. they had a pci-e 2 x16 with 16 ports but it was hard to get.
:looks up and whistles:.....I wouldn't be surprised to see up to 24 port x16 PCIe3 SAS12 cards from LSI Q1-13. That said, retail delivery of said cards may lag at least a quarter or two behind while OEM/Contract implementations come first.
24 port SAS12 ... that is more bandwidth than even pci-e 3 x32 can deliver. 16 port sas12 seems more reasonable at about 20% over what x16 can deliver.
although idk, would be nice for some scenarios. should keep the interrupts lower and you dont 'have' to use all the ports if you don't want too. many of the quad socket boards are shipping with 5 or more x16 slots too which is nice.
although idk, would be nice for some scenarios. should keep the interrupts lower and you dont 'have' to use all the ports if you don't want too. many of the quad socket boards are shipping with 5 or more x16 slots too which is nice.
At this point, x16 3.0 Yields (theoretical here, not real world) 16GB/s. At 24 SSD's, even large sequential transfers averaging 500MB/s you still only hit 12GB/s across the bus. Once higher performance drives start arriving I agree it could be an issue, but very few workflows will put that kind of strain and have the right kind of transfers to really bottleneck the flow, and PCIe4 is just around the corner.
siamsquare
Weaksauce
- Joined
- Oct 30, 2011
- Messages
- 105
Hmmm,
Just made a new pool (raidz) with 4x3TB drives and created a ZFS folder with only 7.8TB usable space? Isn't that bit low?
Just made a new pool (raidz) with 4x3TB drives and created a ZFS folder with only 7.8TB usable space? Isn't that bit low?
It's normal. raidz [or good 'ol HW RAID5] of 4x3tb = 9 TB usable space. Actually its over 8.39 TB-you know,HDD manufacturers counts bytes in different manner from most of type of OSes. Storage medium is measured in decimal ,meaning 1MB=1000 bytes. Operating systems measure the same thing in binary, meaning 1MB=1024 bytes.Add to this " overflow protection (use max 90% of current space)" in napp-it ->Polls->Create Pool "on" at default [which is good] and you have the number
siamsquare
Weaksauce
- Joined
- Oct 30, 2011
- Messages
- 105
It's normal. raidz [or good 'ol HW RAID5] of 4x3tb = 9 TB usable space. Actually its over 8.39 TB-you know,HDD manufacturers counts bytes in different manner from most of type of OSes. Storage medium is measured in decimal ,meaning 1MB=1000 bytes. Operating systems measure the same thing in binary, meaning 1MB=1024 bytes.Add to this " overflow protection (use max 90% of current space)" in napp-it ->Polls->Create Pool "on" at default [which is good] and you have the number
OK thanks,
Just thought 4x3TB=12TB - 3TB(Raidz)= a bit less then 9TB and the 10% is disabled so where is that 1TB gone then???
siamsquare
Weaksauce
- Joined
- Oct 30, 2011
- Messages
- 105