10 Gb Connection

_Gea said:
The point is to connect server and client directly over a new cat6 cable. A crossover cable always works as you must crossover nic to nic or switch to switch. If one of the nics or switches can do auto-crossover (not all can), you can use a 1:1 patchcable.
Click to expand...

From your experience with the Intel X540 T2 should the 1:1 patchcable work with these NIC's and would a cat6 cable work, I though I had to use cat6A?
 
N Bates said:
From your experience with the Intel X540 T2 should the 1:1 patchcable work with these NIC's and would a cat6 cable work, I though I had to use cat6A?
Click to expand...

I had exactly same issue like yours before, same nic.
Turn off flow control and that's it.
 
Getting better results now, I think this would be the differnce between using the LSI1068 vs the LSI2008 controller and possibly the new mobo and the extr RAM possibly:

Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

PS C:\Users\N> ./iperf3.exe -c 10.10.10.10
Connecting to host 10.10.10.10, port 5201
[ 4] local 10.10.10.20 port 60949 connected to 10.10.10.10 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 800 MBytes 6.71 Gbits/sec
[ 4] 1.00-2.00 sec 826 MBytes 6.93 Gbits/sec
[ 4] 2.00-3.00 sec 824 MBytes 6.91 Gbits/sec
[ 4] 3.00-4.00 sec 820 MBytes 6.88 Gbits/sec
[ 4] 4.00-5.00 sec 826 MBytes 6.93 Gbits/sec
[ 4] 5.00-6.00 sec 816 MBytes 6.84 Gbits/sec
[ 4] 6.00-7.00 sec 827 MBytes 6.94 Gbits/sec
[ 4] 7.00-8.00 sec 825 MBytes 6.92 Gbits/sec
[ 4] 8.00-9.00 sec 828 MBytes 6.94 Gbits/sec
[ 4] 9.00-10.00 sec 815 MBytes 6.84 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 8.02 GBytes 6.89 Gbits/sec sender
[ 4] 0.00-10.00 sec 8.02 GBytes 6.89 Gbits/sec receiver

iperf Done.
PS C:\Users\N> ./iperf3.exe -s
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 10.10.10.10, port 62169
[ 5] local 10.10.10.20 port 5201 connected to 10.10.10.10 port 35209
[ 7] local 10.10.10.20 port 5201 connected to 10.10.10.10 port 50193
[ 9] local 10.10.10.20 port 5201 connected to 10.10.10.10 port 48770
[ 11] local 10.10.10.20 port 5201 connected to 10.10.10.10 port 58289
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-1.00 sec 192 MBytes 1.61 Gbits/sec
[ 7] 0.00-1.00 sec 192 MBytes 1.61 Gbits/sec
[ 9] 0.00-1.00 sec 193 MBytes 1.62 Gbits/sec
[ 11] 0.00-1.00 sec 191 MBytes 1.61 Gbits/sec
[SUM] 0.00-1.00 sec 767 MBytes 6.44 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 1.00-2.00 sec 204 MBytes 1.72 Gbits/sec
[ 7] 1.00-2.00 sec 201 MBytes 1.69 Gbits/sec
[ 9] 1.00-2.00 sec 202 MBytes 1.69 Gbits/sec
[ 11] 1.00-2.00 sec 200 MBytes 1.68 Gbits/sec
[SUM] 1.00-2.00 sec 808 MBytes 6.77 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 2.00-3.00 sec 208 MBytes 1.75 Gbits/sec
[ 7] 2.00-3.00 sec 209 MBytes 1.75 Gbits/sec
[ 9] 2.00-3.00 sec 207 MBytes 1.74 Gbits/sec
[ 11] 2.00-3.00 sec 206 MBytes 1.73 Gbits/sec
[SUM] 2.00-3.00 sec 831 MBytes 6.97 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 3.00-4.00 sec 207 MBytes 1.73 Gbits/sec
[ 7] 3.00-4.00 sec 210 MBytes 1.76 Gbits/sec
[ 9] 3.00-4.00 sec 207 MBytes 1.73 Gbits/sec
[ 11] 3.00-4.00 sec 209 MBytes 1.75 Gbits/sec
[SUM] 3.00-4.00 sec 833 MBytes 6.99 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 4.00-5.00 sec 208 MBytes 1.75 Gbits/sec
[ 7] 4.00-5.00 sec 207 MBytes 1.74 Gbits/sec
[ 9] 4.00-5.00 sec 207 MBytes 1.73 Gbits/sec
[ 11] 4.00-5.00 sec 207 MBytes 1.74 Gbits/sec
[SUM] 4.00-5.00 sec 829 MBytes 6.96 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 5.00-6.00 sec 196 MBytes 1.64 Gbits/sec
[ 7] 5.00-6.00 sec 196 MBytes 1.65 Gbits/sec
[ 9] 5.00-6.00 sec 197 MBytes 1.65 Gbits/sec
[ 11] 5.00-6.00 sec 196 MBytes 1.65 Gbits/sec
[SUM] 5.00-6.00 sec 785 MBytes 6.59 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 6.00-7.00 sec 212 MBytes 1.78 Gbits/sec
[ 7] 6.00-7.00 sec 205 MBytes 1.72 Gbits/sec
[ 9] 6.00-7.00 sec 211 MBytes 1.77 Gbits/sec
[ 11] 6.00-7.00 sec 208 MBytes 1.75 Gbits/sec
[SUM] 6.00-7.00 sec 837 MBytes 7.02 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 7.00-8.00 sec 210 MBytes 1.76 Gbits/sec
[ 7] 7.00-8.00 sec 210 MBytes 1.76 Gbits/sec
[ 9] 7.00-8.00 sec 211 MBytes 1.77 Gbits/sec
[ 11] 7.00-8.00 sec 210 MBytes 1.76 Gbits/sec
[SUM] 7.00-8.00 sec 840 MBytes 7.05 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 8.00-9.00 sec 203 MBytes 1.70 Gbits/sec
[ 7] 8.00-9.00 sec 212 MBytes 1.78 Gbits/sec
[ 9] 8.00-9.00 sec 203 MBytes 1.71 Gbits/sec
[ 11] 8.00-9.00 sec 211 MBytes 1.77 Gbits/sec
[SUM] 8.00-9.00 sec 829 MBytes 6.95 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 9.00-10.00 sec 211 MBytes 1.77 Gbits/sec
[ 7] 9.00-10.00 sec 211 MBytes 1.77 Gbits/sec
[ 9] 9.00-10.00 sec 209 MBytes 1.75 Gbits/sec
[ 11] 9.00-10.00 sec 205 MBytes 1.72 Gbits/sec
[SUM] 9.00-10.00 sec 836 MBytes 7.01 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 10.00-10.00 sec 0.00 Bytes 0.00 bits/sec
[ 7] 10.00-10.00 sec 0.00 Bytes 0.00 bits/sec
[ 9] 10.00-10.00 sec 0.00 Bytes 0.00 bits/sec
[ 11] 10.00-10.00 sec 0.00 Bytes 0.00 bits/sec
[SUM] 10.00-10.00 sec 0.00 Bytes 0.00 bits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-10.00 sec 0.00 Bytes 0.00 bits/sec sender
[ 5] 0.00-10.00 sec 2.00 GBytes 1.72 Gbits/sec receiver
[ 7] 0.00-10.00 sec 0.00 Bytes 0.00 bits/sec sender
[ 7] 0.00-10.00 sec 2.01 GBytes 1.72 Gbits/sec receiver
[ 9] 0.00-10.00 sec 0.00 Bytes 0.00 bits/sec sender
[ 9] 0.00-10.00 sec 2.00 GBytes 1.72 Gbits/sec receiver
[ 11] 0.00-10.00 sec 0.00 Bytes 0.00 bits/sec sender
[ 11] 0.00-10.00 sec 2.00 GBytes 1.71 Gbits/sec receiver
[SUM] 0.00-10.00 sec 0.00 Bytes 0.00 bits/sec sender
[SUM] 0.00-10.00 sec 8.00 GBytes 6.87 Gbits/sec receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
 
dilidolo said:
I had exactly same issue like yours before, same nic.
Turn off flow control and that's it.
Click to expand...

I thought I will revisit the flow control settings again as initially when I have checked this it says it is turned off by default, however, I went in the ixgbe driver and try and force it, by iputting "flow_control=0;" and tried "flow_control=0" neither have made any difference, not sure whether I am executing this correctly, I have added the commented out "flow_control=0" under flow control ixgbe driver.

One thing I have still to try out is to replace the cable, I will get some cable ordered today as I have run out and try this next.
 
Update:

Replaced cable and the result is exactly the same so it is not the cable: I have tried the below performace settings without any success:

Windows side:
Interrupt moderation: Disabled
Jumbo Packets: Disabled and 9014
Maximum Nuber of RSS Queues: 8
Flow Control: Disabled, RX & TX enabled, RX only and TX only
Interrupt Moderation Rete: Adaptive and Off
Receive Buffer: Default 512 and 4096
Sending Buffer: Default 512 and 16384

Omnnios Side (I have deleted the adapter before making changes to the /kernel/drv/ixgbe.conf file):
Intr_Throttling: =0 and =1 not sure what the default is.
Flow Control: Default=0, =1, =2, =3


Still can only manage 160 to 180, with some of the above chages it increased for a maybe a second to 380 and went back down again, so I am sure it is a setting somewhere that I have not managed to set correctly.
Anymore suggestions?
 
You must log in or register to reply here.
Back
Top