Fresh Windows Install Gigabit Internet Issue

L

LodeRunner

Gawd
Joined
Sep 8, 2006
Messages
746
I was recently forced to do the first fresh install of Windows 10 on my PC in years (it might be the second time since Windows 10 came out?) and for the life of me I can't remember what network tuning I did to get consistent gigabit results in speed tests (usually fast.com).

iPerf from my desktop to a local server hits 10 Gbit with no fiddling; other systems on the network going to fast.com are reporting 950+ up/down (symmetric fiber), but my newly rebuilt desktop is struggling to get past 500. Network card is a Mellanox CX3 (MCX311A_XCAT). Speed test results are consistent between Edge and Firefox on this machine. NIC drivers are the latest version from Nvidia (who owns Mellanox now) and not the MS packaged drivers; these are the same drivers I used pre-rebuild.
 
go into device manager and set it to 1gb duplex instead of auto. its what i had to do after rogers merged with shaw up here and i started having low speeds on my gb connection.
 
pendragon1 said:
go into device manager and set it to 1gb duplex instead of auto. its what i had to do after rogers merged with shaw up here and i started having low speeds on my gb connection.
Click to expand...
It's a 10 Gbit fiber card and there is no speed/duplex setting for it.
Steam just hit 88 Mbyte/s downloading; local network transfers exceed 1 GB/s:
1718214323298.png

I'd say it was a browser problem rather than a general network problem, but the performance is the same in different browsers (FF and Edge) with completely different sets of extensions (basically none in FF right now since it's a clean install). Pretty sure it's not the firewall since the same site (fast.com) returns the results I expect from other computers on the same network.
 
pendragon1 said:
meant 10gb, just typod. thats odd that is doesnt have it. have you tried speedtest.net?
Click to expand...
What I wrote before rebooting the PC to clear pending updates and heading off to lunch: Similar results: below 500 mb in both directions. Maybe if it was a copper 10Gb card or I used a copper SFP+ it might have speed/duplex settings. Or this is a Mellanox thing where they don't bother to expose it. Changing Interrupt Moderation to disabled made no change; I've definitely had that be a culprit in poorly performing Intel NICs before.

After coming back: of course everything is fine now. I suspect, but can't prove, that the tuning settings that I applied in the Mellanox panel had not been fully applied, but it's exceedingly odd that it would so directly affect speed test results in a browser while not having any impact whatsoever on iPerf and SMB transfers.

For reference for anyone using Mellanox CX3 drivers in Windows, this panel:
1718220070471.png


Edit: and not 5 minutes later, browsers are back to sub-500 mbit throughput to same test sites.
 
pendragon1 said:
meant 10gb, just typod. thats odd that is doesnt have it. have you tried speedtest.net?
Click to expand...
It is not odd at all. No copper 1Gb or higher adapter should ever be set at anything other than auto per spec. I've never seen any fiber NIC with such settings.
 
LodeRunner said:
What I wrote before rebooting the PC to clear pending updates and heading off to lunch: Similar results: below 500 mb in both directions. Maybe if it was a copper 10Gb card or I used a copper SFP+ it might have speed/duplex settings. Or this is a Mellanox thing where they don't bother to expose it. Changing Interrupt Moderation to disabled made no change; I've definitely had that be a culprit in poorly performing Intel NICs before.

After coming back: of course everything is fine now. I suspect, but can't prove, that the tuning settings that I applied in the Mellanox panel had not been fully applied, but it's exceedingly odd that it would so directly affect speed test results in a browser while not having any impact whatsoever on iPerf and SMB transfers.

For reference for anyone using Mellanox CX3 drivers in Windows, this panel:
View attachment 659458

Edit: and not 5 minutes later, browsers are back to sub-500 mbit throughput to same test sites.
Click to expand...
what options are on the advanced tab. IF it has duplex mode it will be on that tab under the properties list or maybe you can do it via cli.
 
The Mellanox driver does not expose Speed & Duplex:
1718226449132.png

Interestingly, on my servers at home and work with Broadcom/QLogic and Intel SFP+ cards, which are using a mix of optics and DACs do expose that setting. The Intel and Broadcom drivers have Auto, 10g full, and 1g full options; the QLogic driver doesn't offer auto, just has 10g and 1g modes.
 
What did you originally do when tuning the card? Did you run into this issue the first time around?
 
What are you using for switches and a router? It's not uncommon for switches and routers to have issues with speed mismatches, especially if they weren't designed for the speeds devices they're attached to operate at.

My first guess would be to check the flow control settings on the NIC.
 
zandor said:
What are you using for switches and a router? It's not uncommon for switches and routers to have issues with speed mismatches, especially if they weren't designed for the speeds devices they're attached to operate at.

My first guess would be to check the flow control settings on the NIC.
Click to expand...
Well, that's the fun bit. Nothing on this computer changed other than wiping the hard drive and reinstalling Windows.

MCX311A-XCAT > 10 Gb optic > Arista DCS-7050QX-32S with the port in 4x10 breakout mode. Router is an OPNsense VM which has not had a configuration change in months. And other systems (server on 10Gb fiber and VMs over 40Gb DAC) are not exhibiting the slowdown in browser. File transfers and iPerf run at line rate from this workstation to other local clients. I'm just at an utter loss why browsing is messed up. Steam downloads from the local CDN were running at 800 Mbps+ (not as high as normal; I've gotten 980 Mbps from Steam), but still that's higher than fast.com, speedtest.net and grabbing an Exchange ISO from Microsoft. ~40 Mbyte/s on the ISO from MS to workstation, 90 Mbyte/s to one of the VM hosts.

I think I'm going to have to try restoring a backup of the workstation in Veeam as a VM and go through my registry settings for network tweaks or something.
 
SamirD said:
What did you originally do when tuning the card? Did you run into this issue the first time around?
Click to expand...
That’s the problem; the configuration I had to blow away had been unchanged for over 5 years, so I really can’t recall what I may have had to do the first time around.
 
LodeRunner said:
That’s the problem; the configuration I had to blow away had been unchanged for over 5 years, so I really can’t recall what I may have had to do the first time around.
Click to expand...
Bummer. :( That's why I'll always make a baseline image of an install once I have it set up. Then I can just restore the image and get back to it.
 
SamirD said:
Bummer. :( That's why I'll always make a baseline image of an install once I have it set up. Then I can just restore the image and get back to it.
Click to expand...
I thought I was being smart having my Veeam server back it up. Well, apparently the workstation RAM was bad and while the backups completed allegedly with no errors, they were corrupted in flight. Veeam default maintenance schedule is once a month. I had 8 days of backups, all bad in different blocks when trying a bare metal restore. File level recovery worked with no reported errors, but who knows until I open a given file. I can't remember if I retried the bare metal restore since I swapped the memory, so I'm may try taking a Macrium image the running environment, then attempt another bare metal restore.

Veeam is now configured to do backup maintenance and validation every night and I have been properly chastised.

I also have an old Macrium image from my last SSD swap (controller got cooked). That’s the better part of a year old.
 
LodeRunner said:
I thought I was being smart having my Veeam server back it up. Well, apparently the workstation RAM was bad and while the backups completed allegedly with no errors, they were corrupted in flight. Veeam default maintenance schedule is once a month. I had 8 days of backups, all bad in different blocks when trying a bare metal restore. File level recovery worked with no reported errors, but who knows until I open a given file. I can't remember if I retried the bare metal restore since I swapped the memory, so I'm may try taking a Macrium image the running environment, then attempt another bare metal restore.

Veeam is now configured to do backup maintenance and validation every night and I have been properly chastised.

I also have an old Macrium image from my last SSD swap (controller got cooked). That’s the better part of a year old.
Click to expand...
Ah yes, this type of stuff scares me. I only have a clonezilla image and who knows what will happen if something gets bit flipped or a ram issue like you ran into. I've only had a problem restoring one image and I think it's because I imaged the wrong drive from the get go, so that supermicro awaits restoration.
 
Well, I have no explanation for this, as the other computers tested were tested while also using RDP, but the browser speed issue appears to somehow be tied to when RDP has been used. I VNC'd in this morning and speeds were 900+ in both directions. RDP'd to same system, speeds immediately cut in half. Disconnected RDP, reconnected VNC, and issue persists, but yesterday, without a reboot, when I had disconnected RDP at work and sat down at home an hour later, it sorted itself out.

I thought I might be a victim of netsh autotuning, but enabled or disabled it made no difference and disabling the autotuning while in RDP also did not fix it.

SamirD said:
Ah yes, this type of stuff scares me. I only have a clonezilla image and who knows what will happen if something gets bit flipped or a ram issue like you ran into. I've only had a problem restoring one image and I think it's because I imaged the wrong drive from the get go, so that supermicro awaits restoration.
Click to expand...
In addition to the maintenance scan in Veeam, I've setup SureBackup jobs to do CRC verifications of all jobs going forward. I'm debating setting up the full virtual lab so that SureBackup can perform actual boot tests.
 
Interesting. What resolution and color depth are you connecting to via RDP? Try 16-bit color vs 32-bit and see if that changes anything. I know fast, et al are very video intensive so they do get hurt by RDP.

That's pretty neat you can do boot tests if fully set up. Eventually I want to virtualize everything I have into a promox ha cluster using the collection of servers I have. Just another project in the midst of other stuff and life though...
 
SamirD said:
Interesting. What resolution and color depth are you connecting to via RDP? Try 16-bit color vs 32-bit and see if that changes anything. I know fast, et al are very video intensive so they do get hurt by RDP.

That's pretty neat you can do boot tests if fully set up. Eventually I want to virtualize everything I have into a promox ha cluster using the collection of servers I have. Just another project in the midst of other stuff and life though...
Click to expand...
1920*1080 @ 32bpp. I dropped it to 24 bpp and 16 bpp and surprisingly, the speed test results got worse of all things. Just to make sure it somehow wasn't the Windows 11 RDP client, I tested from a Windows 10 system with the same results.
 
LodeRunner said:
1920*1080 @ 32bpp. I dropped it to 24 bpp and 16 bpp and surprisingly, the speed test results got worse of all things. Just to make sure it somehow wasn't the Windows 11 RDP client, I tested from a Windows 10 system with the same results.
Click to expand...
So weird! So if you're at the local console, it's fine, but over rdp speed is the issue, right?
 
Is windows still doing the thing where if it thinks you're doing multimedia, it messes up networking; and does RDP count as multimedia?
 
LodeRunner said:
And yeah, local console session, no speed issue in browser. So some weirdness with RDP that I have not previously witnessed or that I have simply forgotten.
Click to expand...
Very interesting. And downloads are at full speed from the console? Like they are on the other systems that are working fine?
 
toast0 said:
Is windows still doing the thing where if it thinks you're doing multimedia, it messes up networking; and does RDP count as multimedia?
Click to expand...
I will double check because I did see that and I did do some tuning which included changing those particular settings. System uptime is over a day, so clearly I haven't rebooted since adjusting and those are registry settings.
 
toast0 said:
Is windows still doing the thing where if it thinks you're doing multimedia, it messes up networking; and does RDP count as multimedia?
Click to expand...
Good point as I always set my rdps to either leave sound at the source or not play it.
 
And yup, it was the ThrottlingIndex and SystemResponsiveness settings under HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Multimedia\SystemProfile. Once I set those properly and rebooted, the problem went away. So yeah, most likely is the audio over RDP; this is my workstation/gaming computer and I RDP in from work and listen to videos or music. Not the most efficient way, but does mean that I don't have to sign into personal accounts, etc. for stuff.

I must have tuned those years ago. I'm trying to think if I ever had to do a repair or full reinstall of Windows 10 to this point, because if I haven't, then the install that got borked by bad RAM was my original Windows 7 install, which had been performed on an AMD Opteron mainboard, moved to an i7-920 where it lived until ~2018/19 and had been in place upgraded from 7 through 8, 8.1, and finally 10, moved from the i7-920 to a Dell T5810 with a E5 Xeon in it, and finally moved to the current AMD 5800X build.
 
You must log in or register to reply here.
Back
Top