TheLastBoyscout
Limp Gawd
- Joined
- Feb 13, 2011
- Messages
- 142
Hi,
I've implemented an all-in-one sever following _gea's (popular) guide:
OpenSolaris derived ZFS NAS/ SAN (Nexenta*, OpenIndiana, Solaris Express
However, despite using 5x 600GB VelociRaptor drives in a "RAID10" + spare configuration, the write performance from within the Windows (XP and 7) VMs is simply unacceptable (benchmarks and perceived performance are roughly on par with a 10 year old HDD
), while the performance within the Solaris OS (OI 148) seems to be OK.
There are a few parameters to tweak and I looked at the following:
1. e1000 vs. vmxnet3 network interface (within the Solaris SAN):
While the vmxnet3 network interface much yields faster (peak) throughput, I noticed that the e1000 is much more consistent and provides a better overall average (running HD tune read tests from within the Windows VMs write requires an empty HDD or in the case virtual HDD).
2. Disabling the write synchronization:
This gives write performance quite a boost, but if I understand it correctly, it is as risky as running a HW RAID5 without a battery backup and defeats the purpose of a reliable system.
_gea also posted here that this is not recommended (see post # 401).
Now what puzzles me is the following:
When I copy a 7 GB .iso file directly to the VMstorage on Solaris (shared via NFS and SMB) from outside the all-in-one system, it takes the same amount of time regardless of whether write sync is disabled or not. However when I copy the same file to a share on a Windows XP VM running on the all-in-one, I see about 4x improvement when the sync is disabled.
Conclusion: The write sync has a huge impact on NFS write performance.
What I have not tried is the use of a ZIL, simply because I don't have an SSD at hand and I don't want to buy one to find out that it doesn't improve the situation. Looking for one, I have not found much information on usable (SSD) drives. The ZIL can be small (with the above listed ~1TB storage a 2 GB ZIL should be sufficient), but it needs a "supercapacitor" or similar device to allow its SRAM or DRAM cache to be written to the flash in case of a power loss. Without the supercapacitor we're having the same problem in case of an abrupt power-loss as already state above.
Now finding an SSD with a supercapacitor isn't that easy. I found the Wintec FileMate 8GB SLC and the similarly priced 40 GB Intel 320.
Is a ZIL the way to go to avoid this (NFS?) write bottleneck with this type of application?
Which SSDs do you use for a ZIL and how much did it improve the write performance?
Thanks for helping!
-TLB
I've implemented an all-in-one sever following _gea's (popular) guide:
OpenSolaris derived ZFS NAS/ SAN (Nexenta*, OpenIndiana, Solaris Express
However, despite using 5x 600GB VelociRaptor drives in a "RAID10" + spare configuration, the write performance from within the Windows (XP and 7) VMs is simply unacceptable (benchmarks and perceived performance are roughly on par with a 10 year old HDD
There are a few parameters to tweak and I looked at the following:
1. e1000 vs. vmxnet3 network interface (within the Solaris SAN):
While the vmxnet3 network interface much yields faster (peak) throughput, I noticed that the e1000 is much more consistent and provides a better overall average (running HD tune read tests from within the Windows VMs write requires an empty HDD or in the case virtual HDD).
2. Disabling the write synchronization:
This gives write performance quite a boost, but if I understand it correctly, it is as risky as running a HW RAID5 without a battery backup and defeats the purpose of a reliable system.
_gea also posted here that this is not recommended (see post # 401).
Now what puzzles me is the following:
When I copy a 7 GB .iso file directly to the VMstorage on Solaris (shared via NFS and SMB) from outside the all-in-one system, it takes the same amount of time regardless of whether write sync is disabled or not. However when I copy the same file to a share on a Windows XP VM running on the all-in-one, I see about 4x improvement when the sync is disabled.
Conclusion: The write sync has a huge impact on NFS write performance.
What I have not tried is the use of a ZIL, simply because I don't have an SSD at hand and I don't want to buy one to find out that it doesn't improve the situation. Looking for one, I have not found much information on usable (SSD) drives. The ZIL can be small (with the above listed ~1TB storage a 2 GB ZIL should be sufficient), but it needs a "supercapacitor" or similar device to allow its SRAM or DRAM cache to be written to the flash in case of a power loss. Without the supercapacitor we're having the same problem in case of an abrupt power-loss as already state above.
Now finding an SSD with a supercapacitor isn't that easy. I found the Wintec FileMate 8GB SLC and the similarly priced 40 GB Intel 320.
Is a ZIL the way to go to avoid this (NFS?) write bottleneck with this type of application?
Which SSDs do you use for a ZIL and how much did it improve the write performance?
Thanks for helping!
-TLB