Slow Network, iPerf is Extremely Low

F

FrostBite

[H]ard|Gawd
Joined
Jun 2, 2004
Messages
1,086
I'm not sure what is going on. I'm going to be building a NAS soon, and while I know my network is not at its best, I finally ran iPerf and I was stunned at how bad it was.

Here is what I currently use below:
Router: DLink DGL-4300 with latest firmware
Computer 1: GIGABYTE GA-G33M-DS2R (Realtek RTL8111B) with latest drivers
Computer 2: ASUS M3N78-VM (Realtek 8211CL) with old drivers (haven't updated new ones)
Cables: Cat6, 14 foot and 50 foot

This is TCP over 20 seconds and the output is in MBytes.
bin/iperf.exe -c 192.168.0.118 -P 1 -i 1 -p 5001 -f M -t 20
------------------------------------------------------------
Client connecting to 192.168.0.118, TCP port 5001
TCP window size: 0.01 MByte (default)
------------------------------------------------------------
[3860] local 192.168.0.131 port 1730 connected with 192.168.0.118 port 5001
[ ID] Interval Transfer Bandwidth
[3860] 0.0- 1.0 sec 11.3 MBytes 11.3 MBytes/sec
[3860] 1.0- 2.0 sec 11.4 MBytes 11.4 MBytes/sec
[3860] 2.0- 3.0 sec 11.5 MBytes 11.5 MBytes/sec
[3860] 3.0- 4.0 sec 11.4 MBytes 11.4 MBytes/sec
[3860] 4.0- 5.0 sec 11.6 MBytes 11.6 MBytes/sec
[3860] 5.0- 6.0 sec 11.4 MBytes 11.4 MBytes/sec
[3860] 6.0- 7.0 sec 11.5 MBytes 11.5 MBytes/sec
[3860] 7.0- 8.0 sec 11.2 MBytes 11.2 MBytes/sec
[3860] 8.0- 9.0 sec 11.7 MBytes 11.7 MBytes/sec
[3860] 9.0-10.0 sec 11.4 MBytes 11.4 MBytes/sec
[3860] 10.0-11.0 sec 11.4 MBytes 11.4 MBytes/sec
[3860] 11.0-12.0 sec 11.7 MBytes 11.7 MBytes/sec
[3860] 12.0-13.0 sec 11.5 MBytes 11.5 MBytes/sec
[3860] 13.0-14.0 sec 11.6 MBytes 11.6 MBytes/sec
[3860] 14.0-15.0 sec 11.5 MBytes 11.5 MBytes/sec
[3860] 15.0-16.0 sec 11.2 MBytes 11.2 MBytes/sec
[3860] 16.0-17.0 sec 10.7 MBytes 10.7 MBytes/sec
[3860] 17.0-18.0 sec 10.7 MBytes 10.7 MBytes/sec
[3860] 18.0-19.0 sec 11.1 MBytes 11.1 MBytes/sec
[3860] 19.0-20.0 sec 11.6 MBytes 11.6 MBytes/sec
[ ID] Interval Transfer Bandwidth
[3860] 0.0-20.0 sec 227 MBytes 11.4 MBytes/sec
Done.

Here's UDP, same parameters.
bin/iperf.exe -c 192.168.0.118 -u -P 1 -i 1 -p 5001 -f M -b 1.0M -t 20 -T 1
------------------------------------------------------------
Client connecting to 192.168.0.118, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size: 0.01 MByte (default)
------------------------------------------------------------
[3860] local 192.168.0.131 port 1744 connected with 192.168.0.118 port 5001
[ ID] Interval Transfer Bandwidth
[3860] 0.0- 1.0 sec 0.12 MBytes 0.12 MBytes/sec
[3860] 1.0- 2.0 sec 0.12 MBytes 0.12 MBytes/sec
[3860] 2.0- 3.0 sec 0.12 MBytes 0.12 MBytes/sec
[3860] 3.0- 4.0 sec 0.12 MBytes 0.12 MBytes/sec
[3860] 4.0- 5.0 sec 0.12 MBytes 0.12 MBytes/sec
[3860] 5.0- 6.0 sec 0.12 MBytes 0.12 MBytes/sec
[3860] 6.0- 7.0 sec 0.12 MBytes 0.12 MBytes/sec
[3860] 7.0- 8.0 sec 0.12 MBytes 0.12 MBytes/sec
[3860] 8.0- 9.0 sec 0.12 MBytes 0.12 MBytes/sec
[3860] 9.0-10.0 sec 0.12 MBytes 0.12 MBytes/sec
[3860] 10.0-11.0 sec 0.12 MBytes 0.12 MBytes/sec
[3860] 11.0-12.0 sec 0.12 MBytes 0.12 MBytes/sec
[3860] 12.0-13.0 sec 0.12 MBytes 0.12 MBytes/sec
[3860] 13.0-14.0 sec 0.12 MBytes 0.12 MBytes/sec
[3860] 14.0-15.0 sec 0.12 MBytes 0.12 MBytes/sec
[3860] 15.0-16.0 sec 0.12 MBytes 0.12 MBytes/sec
[3860] 16.0-17.0 sec 0.12 MBytes 0.12 MBytes/sec
[3860] 17.0-18.0 sec 0.12 MBytes 0.12 MBytes/sec
[3860] 18.0-19.0 sec 0.12 MBytes 0.12 MBytes/sec
[3860] 19.0-20.0 sec 0.12 MBytes 0.12 MBytes/sec
[ ID] Interval Transfer Bandwidth
[3860] 0.0-20.0 sec 2.39 MBytes 0.12 MBytes/sec
[3860] WARNING: did not receive ack of last datagram after 10 tries.
[3860] Sent 1702 datagrams
Done.
 
Increase the window size to 64K with -w64k on the command line and test again. The default windows on Win32 are way too small.

Failing that, are you actually linking at GbE? This looks exactly like 100mbit performance to me, which would be quite lucky if both PCs linked at GbE. And obviously update those drivers. Realtek NICs are notoriously awful and the driver makes a big difference.
 
as much as I love iperf, the udp test results are always wrong (at least for me). Get the tcp ones working right, and I'd ignore the UDP results.

The default window size should give you 200-300mb on a gig connection, but bumping them to 64k (like keenan said) will get you closer to 800-900mb on a good gig connection.

Definitely upgrade any NIC drivers and re-test.
 
See what a difference the window size makes

Code: 
iperf -c wh-mon06
------------------------------------------------------------
Client connecting to wh-mon06, TCP port 5001
TCP window size: 8.00 KByte (default)
------------------------------------------------------------
[1856] local 192.168.220.35 port 2602 connected with 192.168.220.87 port 5001
[ ID] Interval       Transfer     Bandwidth
[1856]  0.0-10.0 sec   241 MBytes   202 Mbits/sec

iperf -w 65535 -c wh-mon06
------------------------------------------------------------
Client connecting to wh-mon06, TCP port 5001
TCP window size: 64.0 KByte
------------------------------------------------------------
[1856] local 192.168.220.35 port 2604 connected with 192.168.220.87 port 5001
[ ID] Interval       Transfer     Bandwidth
[1856]  0.0-10.0 sec   602 MBytes   504 Mbits/sec
 
Yes, I am sure I am on a GbE. I tested the -w 64k before I left this morning and it went from 11MB to 13MB :confused:

I'll go home and update the NIC drivers on the other computer.
 
There's a ton of things this could be. Bad cables, bad NICs, a faulty switch, software issues. Start isolating compoents and see what it takes to get better performance.
 
Do all your devices support Jumbo Frames? Try enabling that and see what you get...although even without jumbo frames you should be getting a bit higher readings.
 
I'm thinking software drivers. If it was a bad NIC/cable, the stuff more than likely wouldn't be working in the first place. The easiest test I can think of is to use a Linux Live CD in both PCs and do the test again. That will tell you if it is software related.
 
I just stopped by Frys and bought new cables Cat6 cables just in case. The router, a Dlink DGL-4300 "GamerLouge" has a 4 port GbE switch, and doing some searching online, works with GbE for sure. It does not support Jumbo frames though.

I guess beyond trying the new cables, I will update the second computers drivers. The first computer drivers and the routers firmware were updated right before the test so I know for sure those are the latest. I'll have to see what the Linux Live D is and do some testing. I would really hate to buy new NICs because I know at least 600Mb/75MB should be easily attainable from both computers PCI (and not PCIe) based NICs. I hope I can figure this out because 13MB is going to make a NAS unrealistic and hopeless.
 
Can you cable the 2 pc's directly together? No crossover needed - gigabit will (should?) auto MDI/MDIX. This way you can eliminate the router/switch from the equation.
 
I came home and intended to run iperf with the old cables and than the new. My first run:

bin/iperf.exe -c 192.168.0.118 -P 1 -i 1 -p 5001 -w 64.0K -f M -t 10
------------------------------------------------------------
Client connecting to 192.168.0.118, TCP port 5001
TCP window size: 0.06 MByte
------------------------------------------------------------
[3860] local 192.168.0.131 port 2017 connected with 192.168.0.118 port 5001
[ ID] Interval Transfer Bandwidth
[3860] 0.0- 1.0 sec 111 MBytes 111 MBytes/sec
[3860] 1.0- 2.0 sec 111 MBytes 111 MBytes/sec
[3860] 2.0- 3.0 sec 111 MBytes 111 MBytes/sec
[3860] 3.0- 4.0 sec 111 MBytes 111 MBytes/sec
[3860] 4.0- 5.0 sec 110 MBytes 110 MBytes/sec
[3860] 5.0- 6.0 sec 111 MBytes 111 MBytes/sec
[3860] 6.0- 7.0 sec 111 MBytes 111 MBytes/sec
[3860] 7.0- 8.0 sec 108 MBytes 108 MBytes/sec
[3860] 8.0- 9.0 sec 110 MBytes 110 MBytes/sec
[3860] 9.0-10.0 sec 110 MBytes 110 MBytes/sec
[3860] 0.0-10.0 sec 1105 MBytes 110 MBytes/sec
Done.

What the hell, it fixed itself ????

I tried to copy an 8 gig movie via explorer and it took roughly 5 minutes or 26MB/sec. Hmm.. time to find a better way to copy files between computers. Just installed Teracopy and I moved a 7.95 file in a bit under 3 minutes or 44.16MB/sec, must better than before.
 
Last edited:
LOL, you might have just overlooked something. Don't you just love it when problems fix themselves? 26MBs is OK; might need faster hard drives?

I know it's not the router/switch. I've got the same one and do 60MB/s transfers all the time.
 
FrostBite said:
I tried to copy an 8 gig movie via explorer and it took roughly 5 minutes or 26MB/sec. Hmm.. time to find a better way to copy files between computers. Just installed Teracopy and I moved a 7.95 file in a bit under 3 minutes or 44.16MB/sec, must better than before.
Click to expand...
What OSes? Windows XP file copy engine sucks. XP's SMB implementation sucks. Do you have a Vista or Win7 system to test against? I get ~45MB/s pulling from unRAID to any of my XP machines, but 80-90MB/s pulling from the same server to my Win7 client. For more on the "Vista difference", see Mark Russinovich's "Inside Vista SP1 File Copy Improvements".
 
My main (Gigabyte) runs XP and the HTPC (Asus) runs Vista. I didn't think about testing vice versa, good plan on that. I hope this changes in a week when I have my Win 7's.

Ughh... Vista to XP even worse than XP to Vista...
bin/iperf.exe -c 192.168.0.131 -P 1 -i 1 -p 5001 -w 64.0K -f M -t 10
------------------------------------------------------------
Client connecting to 192.168.0.131, TCP port 5001
TCP window size: 0.06 MByte
------------------------------------------------------------
[108] local 192.168.0.118 port 49507 connected with 192.168.0.131 port 5001
[ ID] Interval Transfer Bandwidth
[108] 0.0- 1.0 sec 35.1 MBytes 35.1 MBytes/sec
[108] 1.0- 2.0 sec 34.9 MBytes 34.9 MBytes/sec
[108] 2.0- 3.0 sec 34.3 MBytes 34.3 MBytes/sec
[108] 3.0- 4.0 sec 35.3 MBytes 35.3 MBytes/sec
[108] 4.0- 5.0 sec 34.6 MBytes 34.6 MBytes/sec
[108] 5.0- 6.0 sec 35.2 MBytes 35.2 MBytes/sec
[108] 6.0- 7.0 sec 34.9 MBytes 34.9 MBytes/sec
[108] 7.0- 8.0 sec 35.3 MBytes 35.3 MBytes/sec
[108] 8.0- 9.0 sec 35.0 MBytes 35.0 MBytes/sec
[108] 9.0-10.0 sec 34.7 MBytes 34.7 MBytes/sec
[108] 0.0-10.0 sec 349 MBytes 34.9 MBytes/sec
Done.
 
WTF? Your iperf results shouldn't change that much since they're just tcp streams from nic to nic. With 64k tcp windows you should see 900Mbs. PCI nics might be a little slower, but not that slow.
 
Well, just updated the LAN drivers on the other computer and I'm still getting 35MB/sec. Back to the drawing board.
 
There's likely not a lot you can do. Try the direct connection and see if that makes things better, but I kinda doubt it will. Maybe fiddle with the TCP tunables (receive window and so on) with TCP optimzer or something.

I've had enough random issues with those Realtek pieces of shit to just chalk it up to that, though.
 
Do you have a spare NIC? I own a few Intel PCI and PCIe NICs and they've been flawless in every application. They're supported by nearly everything. Your Asus mobo has a PCIe x1 slot - Intel's PCIe NIC is $30.
 
You must log in or register to reply here.
Back
Top