• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Slow network performance issue with OpenIndiana.

S

Silenus

Limp Gawd
Joined
Jan 6, 2011
Messages
177
I've got a slow network performance issue with OpenIndiana happening and I need some help trying to figure what is going on.

Intel S1200BTS board, E3-1230 CPU, Intel Pro/1000 CT (82574L) NIC.

Main pool is 8 disk RAIDz2 of 2TB Samsung Spinpoint F4's.I had recently migrated over from a Nexenta setup where this same exact storage pool easily pegged gigabit speed in both directions. I have moved over to OI 151a with running NappIt 0.6i currently. I am running physical with static IP's, not nwam. OI is using e1000g driver as expected. Bonnie++ shows 520MB/s write, 670MB/s read. There is plenty of sequential performance on the pool which I knew from the previous setup.

Problem....doing large sequential read and write to the pool I can only get about 40MB/s read and write over the local network. I have also eliminated all switches and connected my machine directly to the above mentioend NIC with exactly the same results. Something is slowing this down and I am not sure what to look at next.

EDIT for clarification!!:
The same storage pool was used on the nexenta install, but NOT the same hardware. The system was previously installed on a Supermciro X8SIE board and Xeon X3430. However, that board too uses the 82574L NIC and I previously was get line speed gigabit on both read and write from this pool. I have also confirmed that this same behavior is happening on BOTH the NIC's I am using on the new system. I also have a Pro/1000 CT card installed which is again the same 82574L NIC. It is behaving as if something is artificially throttling the network speeds in and out.
 
Last edited:
Are you using a 3rd party file copy tool?

If you use IOMeter and do 2MB sequential reads with a qd of 4 do you get more than 40MBps?
 
No third party. Just Windows copy and paste. Crystal Disk sequential test shows the same. Other computers show the same. I haven't run IOmeter but I am sure it will also show the same.

Edit: CrystalDiskMark showing 18 MB/s Write, 39MB/s Read. Hmm, the plot thickens!
 
Last edited:
Hi,

I am building similar setup. When I`ll be done, I`ll do some measures to compare.

Kiwwiaq
 
Are transfering directly from PC to the Server? I get throttling when i use my Desktop and transfer from 2nd pc to the server.
 
This is direct from pc to nic on the NAS. However it is the same even though my usual connection is through 2 switches to the NAS. Makes no difference either way.
 
Install Wireshark and take a packet capture while copying a 1MB test file...IM me and I'll analyze it for you.

I'm confused by your bonnie results...GigE has capacity to do approx 100-120MB/s...you quoted 500MB/s...I presume you mean 500Mb/s which is 40MB/s?

A couple of things to check:

1) Check server and client network adapter...see what speed and duplex settings the adapters have negotiated to.

2) Check adapter *Ethernet* flow control settings...IME I have found disabling this on both client and server can yield better results

3) You using a Cat 5 or Cat5e cable?

4) Did you say you eliminated the switch and connected client directly to server nic using Ethernet cross-over cable?
 
Packetboy:

NAS is at work. I will be back there on Monday and go over these things again and run the wireshark test. However I can tell you I already checked connections at 1000 full duplex, and yes eliminated the switches for direct connect. However, again....even with the switches in place and the same cables....the previous setup could go line speed gigabit from my workstation no problem.

BTW! I also have a small SAN built on the same exact OS install and also with NappIt 0.6i. However the SAN has Intel 82576 NIC's and OI installs the ibg driver for them. I can tell you that connecting to a share on the SAN over the management NIC, with he same cables and through the same switches it goes full gig speed read and write! Different NIC and driver is the only significant difference (that I've noticed so far).
 
Last edited:
Did you try checking the MTU setting on both interfaces? With my 10gbe cards switching the MTU on Win 7 from 1500 to 9000 doubled the performance (solaris default was 9000 for my 10gbe card).
 
Sounds like no Jumbo Frames. Try MTU 9000 as mikeo suggested. GigE seems to only do about 40MB a second with MTU at 1500.
 
Modern machines can saturate Gigabit EASILY even with 1500 MTU.

It only becomes an issue when stepping up to 10Gb or more.
 
Some NICs can and some can't in my experience. Usually lower end stuff, like my Atom based ZBox I used as an HTPC, I had to go to 4500 to get it to keep up to the WD Scorpio Black in it. It wouldn't go over 40MB a second or so at 1500 but at 4500 (highest it supports) full speed of the drive (90ish) was attained.
 
Recent research has shown Jumbo frames don't really have much of an affect anymore, at least on server grade NICs.

http://www.boche.net/blog/index.php...mparison-testing-with-ip-storage-and-vmotion/

http://codingrelic.geekhold.com/2011/12/requiem-for-jumbo-frames.html

I've got two NICs, and it makes a huge difference in throughput on my FreeNAS array depending on which one I use. I can get 40 MB/s on a crappy Marvel desktop NIC. I replace it with a server Intel Pro 1000 I got off eBay for $40, and it goes up to 90 MB/s. All at 1500 byte MTU.
 
You must log in or register to reply here.
Back
Top