How to get more out of Gigabit Ports

B

balance101

Limp Gawd
Joined
Jan 31, 2008
Messages
411
Hi there I recently build a NAS/SAN

Intel Xeon 1260L
Intel S1200KP
Kingston 2x8GB 1333 Mhz ECC Memory
Silverstone ST50F 500W with short cable
Lian Li Q25

IO 151a5 w/ nappit
zraid2
10 x HD204UI 2 TB
IBM M1015 IT mode
2 x Crucial M4 64 GB

I'm hitting the 100MB/s, the motherboard has dual gigabit ports. Being a noob I tho I could some how bind them together to increase my throughput but my friend let me know that was not physically possible. He said I could segregate the data that goes through each port like one for management and other for data.

Also my main rig also have dual gigabit ports, wonder if I can use that to increase my throughput from this computer to the newly build NAS

ATM the NAS and my main computer is connected to a Netgear 3500L (tomatousb Firmware) on wireless bridge mode. Main router is a netgear wndr3700. Also have 2 computers using wireless network to access the NAS

So I google around and came to few hits

Full Duplex - can increase speed to 250MB/s ?
does router have to support? and both NIC ? Ethernet cable cat 5/6 ok?

jumbo frames - can increase throughput but have to test which one is optimal? MTU on router needs to match jumbo frames on NIC setting?

10 Gigabit is out of the question, cost too much and I don't have 10 Gigabyte switch

What other option should I be aware of too

Thanks I have no networking experience
 
100MB/S is fantasic on GbE, you typically lose some to overhead. Using both NICs as you describe is called Link Agregation and will not increase speed between only 2 nodes. Your NICs should automatically set to Full Duplex and they are at Full duplex if you are getting 100MB/s.
The router would only have to support GbE if you need to cross collision domains and maintain speed.
Whatever cable you have now is fine, since you are getting the maximum speed, but CAT5e is the minimum. CAT6 would allow you to get maximum speed under challenging conditions.
Jumbo Frames would probably work if enable between the device, switch, and your computer, but Jumbo Frames can be a pain on mixed networks and can cause all kinds of goofy issues.

The only other option would be to use a 10G crossover- direct connection between the NAS and your computer- no switch would be needed.
 
Sorry I don't quite get "10G crossover"

So if I connect the unpopulated Ethernet ports on computer to NAS. I can get more than 100MB/s? :D

thanks
 
You could technically double your download/upload rate, but only under certain conditions and your NIC drivers will need to support Link Aggregation as well as the switch and server. The speed will only be available in separate operations, such as 2 separate downloads. This is all theoretical, I'm not sure how well it will work for you in practical terms.
By 10G crossover, I mean connecting the 10G NICs of two separate computers with a single cable, no switch in between. Crossover refers to a cable where the transmit and receive pairs are switched (they cross-over on a diagram) so two network nodes can communicate (a similar function in serial cable is called a null-modem cable). A patch cable is straight-through. A crossover cable is dangerously close to legacy status, as most switches and every NIC 1Gb or faster will automatically perform this function, so you can use pretty much any 10G rated cable.
 
I hit 121'ish MB/sec on GBit across my network and all networks I build.

The problem with your GIG network is a number of factors. A cheap ass NIC will not hit 99-100% util and a slow hard drive might not read or write fast enough for full throughput.

Also cheap $25-50 switches although built with Gig spec may not have the ASIC power to hit full throughput..

Im using a $5000 Cisco Enterprise switch for my home and believe me the ASIC in this thing is capable of 160gbps switching fabric with all 24 gig ports running at 100% and the two 10G/e ports running at max. This is so far FAR FAR overkill for you. But I recommend a good name brand upper level HOME gig switch incase you are not using a high quality one. My switch: http://www.newegg.com/Product/Product.aspx?Item=N82E16833120447 --secret--- I didn't pay this much hahaha

A really good switch, I used to hate until I figured out how to trunk with them, is Linksys GS series. Very cost effective and my 8 port in my living room is fully capable of 100% utilization.
Here is a link: http://www.newegg.com/Product/Product.aspx?Item=N82E16833122364&Tpk=GS724T and http://www.newegg.com/Product/Product.aspx?Item=N82E16833122381&Tpk=GS108t

Full Duplex means full line speed ingress and full line speed egress at the same time. Not sequential between hosts.

Also Jumbo frames is HORRIBLE. Turn it off completely unless you are running iSCSI connections. PERIOD! Trust me on this. It will slow your network down.

Also linking (binding) teaming network adapters as in 802.3AD aka Link Agregation will not double your throughput speed between two hosts, but it will allow more bandwidth to be offered to when more than one hosts is pulling data from your device. I.e. 3 computers all pulling a file from your nas that has two nics in team mode will max at 2gbps to multiple hosts pulling from the nas but will not surpass 1gbos to any one host.
 
Last edited:
Last edited:
You might be able to get a bit more speed by upgrading your NIC on your client machine as well. I've noticed most integrated realtek NICs max out at around 100 where with the same setup and an intel addon NIC you can push a lot closer to the 125 max. I'd spend money there before getting a new router, esp since you didn't say what router/switch you're currently using, even the cheap ones can typically support max transfer speeds on two or three links at a time, only running into problems if you're sending and recieving at max on all ports simultaniously.

A little advertised fact about the upcoming windows 8/windows server 2012 is that they naitively support multiple NICs for single file copies (called SMB Multichannel), but would require a win8/ws2012 client and host. I think that at this time this is your only real option for >125MB/s single file copies.
 
It's a bit disheartening to hear that in 2012 they are still selling switches that don't offer full gigabit utilization. I have a Linksys 8-port workgroup switch from 2003 that is still working and that is good for 95%+ gigabit utilization in a SoHo environment. Have we regressed?
 
Hearing so much good things about windows 8

Trim on raid ssd and now this.

Atm i'm looking to see if my current netgear 3500l can support vlan and trunk dual Gigabit on my nas

If not might invest in that 8 port gigabit switch your recommended

Also have to research a bit on how to do link aggressation on console

I have to see if i have any more slot on my client lol sli and sound card and raid card

Just want to confirm for link aggregation i need dual nic with 802.3 ad or q or x?
And switch with vlan and 802.3x ?

Thanks
 
Win8 actually can do the LAG at the OS level, so you would just need 2 NICs. However SMB multichannel doesn't seem to work across a LAG, seems to actually work better just having 2 NICs with 2 different IP addresses.
 
If you want a decent Gig switch for a good price that can do LACP, check out a used HP 1800-24G or 1810-24G.
 
@/usr/home 400~ for a 24 port not bad :D

@-dragon- i want windows 8 :/ take my money M$
 
after searching around I found Realtek Teaming Utility for my lans

I can do LACP on my OI NAS

now I just need a LACP switch :D

HP 1800 does look very temping but do I need 24 port :/

can't decide XD
 
Last edited:
Intel Desktop CT Nics can do all modes of teaming in Windows and Linux. Also can do teaming in many opensource NAS operating systems.
 
@tangoseal

the dlink switch you recommended above does not support 802.3AD ?

The 8 port netgear have bad reviews on newegg =(
 
GS724T is on sale at NCIX 170+ tax mix reviews on newegg

lots of people seems to be happy with their HP 24 ports
 
Of course the dlink doesnt support 802.3ad it is a cheap ass switch. I didn't catch that you required teamable NICs when I made that recommendation. You are probably not going to find 802.3ad in any switch less than $99.00

If you want 802.3ad I recommend you look at a 24 port smart switch like HP or Netgear GS724T.
 
I use TrendNet TEGs80g switches in my home network. Are they cheap, yep, do they work, yep. Understandably with unmanaged switches you don't have the ability to run VLANs, lag groups and many other feature sets that a managed switch will have. Having said that, you typically buy a cheap unmanaged switch because you don't need any of those features. The four Trendnets that I have running in my home network right now have been rock solid for over a year. **knock on wood**

If these devices start to go, the two "core" switches will be replaced by a managed device, just because I am a network engineer by trade and like to have the ability to mess with configs, look at interface counters, logs etc...Truth be told, i have two cisco 2960g switches, one is a 24 port the other a 48 sitting in my basement collecting dust. I used these to study and pass my CCNA. The only problem with implementing one of these bad boys into my network is that all my computer gear is in my family room. Space is limited and these Cisco switches are deep. I tried to mount in many different positions but never found a way to be satisfied with how it turned out. Therefore I kept my little trendnets in production. Depending on what type of files you are transferring, I can hit 50mb but usually I max out at 20mb a second. My desktop is an old Dell media center edition and the read/write of the HDD just cant handle it. The speeds are fine for me as i only transfer files from my desktop to NAS once in a while. I never really tax out my nextwork with file transfers. Sorry for the rant....

IMG_0155.jpg

IMG_0156.jpg
 
Last edited:
so far after looking, I probably get the Netgear GS724T, unless if I can get the HP 1800/1810 for around the same price

Just need to wait for Netgear MIR $75. Right now it would cost me $200 including tax. Come on $75 :D

I just don't want to deal with the netgear GS108T. Another option I thought about was buying another router that supports Tomato/DD-WRT but read that there might be performance issue since its software 802.3AD
 
balance101 said:
so far after looking, I probably get the Netgear GS724T, unless if I can get the HP 1800/1810 for around the same price

Just need to wait for Netgear MIR $75. Right now it would cost me $200 including tax. Come on $75 :D

I just don't want to deal with the netgear GS108T. Another option I thought about was buying another router that supports Tomato/DD-WRT but read that there might be performance issue since its software 802.3AD
Click to expand...

I own a GS108T 8 port Version 1 (The older model) WORKS PERFECT!! But it was a monster to figure how to trunk on this thing. NO documentation.
 
how does the Cisco SG 200 - SLM2024T-NA compare?

its price between the netgear and HP

http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps11229/data_sheet_c78-634369.html

http://www.newegg.com/Product/Product.aspx?Item=N82E16833150121

Nov 10, 2011 3:04 PM (in response to David Hornstein)
lacp groups supported on sg200-26

It has not been changed as of yet. I bought this exact switch because i was led to belive that this would support 8 LAG groups only to find out after receiving it that it only supports 4 LAG groups. This is not for home use and i need more than 4 LACP LAG groups. So now i either have to go through the hassle of returning it or try to figure out a way to use this switch with our other switches that do not support LACP.



Please update your data sheet ASAP as it has another victim in me. Again the main reason i bought this switch was that i thought it supported more than 4 LACP LAG groups.



Support for IEEE 802.3ad Link Aggregation Control Protocol (LACP)

#

• Up to 8 groups

#

• Up to 4 ports per group with 16 candidate ports for each (dynamic) 802.3ad link aggregation



http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps11229/data_sheet_c78-634369.html
Click to expand...

https://supportforums.cisco.com/thread/2109486
 
As an eBay Associate, HardForum may earn from qualifying purchases.
I've used the SG200's they're definitely solid units, just not layer 3 which is why I upgraded.
 
A used Dell PowerConnect 5224 or newer (5324, 5424) will do everything you need for as low as $50. The SG200 series is a great switch with really good firmware and interface. There is no serial connection, but I haven't run into an issue because of it. Other smart or managed switches without a serial port should be a red-flag.
 
the Foundry FGS is pretty expensive =( looking for cheapest 802.3AD Switch $100-300

wow a used DELL PowerConnect 5424 24port Gigabit Managed Layer 2 Switch 1U Rack Ears WRNTY is only $300 on ebay =O

from cheapest

Netgear GS724T - 125 (if MIR available) new
Dell PowerConnect 5224 - 150 used
Dell PowerConnect 5324 - 250 used
HP 1800 - 250 used
Cisco GS200 24 - 280 new
Dell PowerConnect 5424 - 300 used
HP 1810 - 340 new

"Link Aggregation with support for up to 8 aggregated links per switch and up to 8 member ports per aggregated link (IEEE 802.3ad)"

what does this mean? like how many groups max? 4 ? on 24 port?
 
Last edited:
Even if there's a problem the Cisco Small Business support is pretty fantastic from my experience.

And that means you can have up to 8 total LAG's with up to 8 ports each, so if you go for max ports then you can only have 3 LAGs or use 16 ports for 8 2 port LAGs
 
IC Dragon

If I go for max LAG groups (being 8 groups) so 3 ports per group? Powerconnect is temping but do I need all the LAG Groups
 
8 LAG groups would require 8 different machines with at least 3 NICs each, so I highly doubt you'll use all 8. Honestly LAGs in single or low number user environments are pretty much useless, the SMB multistream feature I talked about earlier requires one IP per stream so for windows NT 6.2 systems with single user work loads, a 4 port LAG would actually be slower than 4 individual NICs with 4 individual IPs. I've tried LAGs of various sizes on my network from 2-5Gbit and never been that impressed with them, hopefully W8/WS'12 with multiple IPs will actually let me see 150+ transfers
 
Already some good feedback here. tangoseal definitely hit the bullseye in explaining why Cisco enterprise switches cost more, the ASIC is a very good example why. For the budget you have available, I think you're probably looking at the best you can get.

One thing though is that you don't mention exactly what you're doing that you need more performance. I'm a little confused why you are bringing your router into the equation unless you are specifically copying data to/from your local nas to some destination on the internet.

For your LAN, you should already be at full-duplex on everything. Most all equipment is set to auto-negotiate so unless something is wrong it should already be there.

Jumbo frames I would only recommend for large sustained file transfers. Earlier this year was the first time I enabled this stuff at my work and that was only after we slapped in our new 10Gig blades. I had thought about it at 1 gig, but decided against because I wasn't so sure I'd see a real noticeable impact. But with it enabled now, we should begin seeing backup times drop across the board...I'm still waiting for new equipment I ordered for our SAN to arrive so I don't have any conclusive results to share at the moment.

So unless you are transferring gigs upon gigs of data consistently across your LAN, you probably won't realize any advantage with jumbo frames.

One thing you might check is if your o/s & nic drivers support receive side scaling. If your machine has multiple cores or processors, this will allow the nic to utilize multiple cycles across all cpu's instead of just cpu 0. Again you likely won't notice any real improvement on the small scale you have here...stuff like this is really handy, for example, with the databases we have that are outside facing that are getting pounded all the time with requests.

Another thing to look at is your hard drives. Mechanical hard drives generally top out around 1 gig anyway, so some may even have a hard time maxing out. Have you tried timing a test file? Try and transfer a large test file, say 25GB or whatever, from your workstation to the nas. Record the time. Now transfer the same file from an SSD in your workstation to an SSD on your nas and compare results. You'll probably see some improvement on the SSD side, but I doubt you want a nas full of SSDs.
 
If you want the absolute best of the best in layer 2 switching (Opinion of course) I would look at an (USED) 8 port Cisco 2960G-8TC-L.

Here is an Ebay Link. I know it cost more but you will never look back and hey ... you might learn some real hardcore networking in the process haha...

These support almost every single layer 2 switching protocol on the market. You can get more ports on the HP but then again this is Cisco which is a premium price anyways. These switches retail for about $750'ish brand new just for the 8 port.

http://www.ebay.com/itm/Cisco-Catal...054536387?pt=COMP_EN_Hubs&hash=item3cc8101ac3
 
As an eBay Associate, HardForum may earn from qualifying purchases.
-Dragon- said:
8 LAG groups would require 8 different machines with at least 3 NICs each, so I highly doubt you'll use all 8. Honestly LAGs in single or low number user environments are pretty much useless, the SMB multistream feature I talked about earlier requires one IP per stream so for windows NT 6.2 systems with single user work loads, a 4 port LAG would actually be slower than 4 individual NICs with 4 individual IPs. I've tried LAGs of various sizes on my network from 2-5Gbit and never been that impressed with them, hopefully W8/WS'12 with multiple IPs will actually let me see 150+ transfers
Click to expand...

fair enough, I could only imagine 4 Ports Group. :D

@wixter, when I copy from my computer to my NAS, I get 100-105MB/s. My Computer and NAS is connected by a Netgear 3500L router. Haven't try SSD yet :D

Family members will be access the NAS over wifi N and some over ethernet. (4 computer and some smart phones) I recently started doing lots of game recording like FRAPS etc. Am considering Writing my recording directly to my nas, not sure if that's a good idea.

well with third party firmware like tomato/DD-WRT it can add lots of rich feature to your router. Some DDWRT forum report, LACP working.

@tangoseal CISCO Catalyst 2960 Series WS-C2960-8TC-S Ethernet Switch 10/100Mbps + 1000Mbps 8 x RJ45 + 1 x SFP - $350 any good? :D

Damn Cisco never have MIR or sale ? :|

Anyone hear of Juniper Switch, hear they are competing with Cisco but don't see any 100-500 dollar switches :/
 
Juniper competes more on the medium to high end, not heard of juniper "small business" switches.
 
balance101 said:
fair enough, I could only imagine 4 Ports Group. :D

@wixter, when I copy from my computer to my NAS, I get 100-105MB/s. My Computer and NAS is connected by a Netgear 3500L router. Haven't try SSD yet :D

Family members will be access the NAS over wifi N and some over ethernet. (4 computer and some smart phones) I recently started doing lots of game recording like FRAPS etc. Am considering Writing my recording directly to my nas, not sure if that's a good idea.

well with third party firmware like tomato/DD-WRT it can add lots of rich feature to your router. Some DDWRT forum report, LACP working.

@tangoseal CISCO Catalyst 2960 Series WS-C2960-8TC-S Ethernet Switch 10/100Mbps + 1000Mbps 8 x RJ45 + 1 x SFP - $350 any good? :D

Damn Cisco never have MIR or sale ? :|

Anyone hear of Juniper Switch, hear they are competing with Cisco but don't see any 100-500 dollar switches :/
Click to expand...

It is a good switch BUT ... its 10/100 not full gig except for that uplink port. They are fantastic switches but you are looking for gigabit so I dont know if this is going to serve you well.

Cisco does have rebates but .... they are very specific and only on BRAND new equipment.
 
I think we kinda need to go back to the basics here and ask, what is it EXACTLY you're trying to do here? Most of the stuff we've discussed lately (esp the expensive switches) has more to do with learning more about networking, they'll be of minimal use for your original problem of trying to get much over 100MB/s file transfers. If your desire has moved beyond that to wishing to learn more about IP networking then we'd be remiss in not recommending layer 3 switches, which do cost a bit more but also have way more features.

So the big 3 questions to answer are:

What exactly are you wanting to do now?
How many ports do you need?
What's your budget (and are you ok with used and/or ebay or do you prefer new?)
 
tangoseal said:
I hit 121'ish MB/sec on GBit across my network and all networks I build.

The problem with your GIG network is a number of factors. A cheap ass NIC will not hit 99-100% util and a slow hard drive might not read or write fast enough for full throughput.

Also cheap $25-50 switches although built with Gig spec may not have the ASIC power to hit full throughput..

Im using a $5000 Cisco Enterprise switch for my home and believe me the ASIC in this thing is capable of 160gbps switching fabric with all 24 gig ports running at 100% and the two 10G/e ports running at max. This is so far FAR FAR overkill for you. But I recommend a good name brand upper level HOME gig switch incase you are not using a high quality one. My switch: http://www.newegg.com/Product/Product.aspx?Item=N82E16833120447 --secret--- I didn't pay this much hahaha

A really good switch, I used to hate until I figured out how to trunk with them, is Linksys GS series. Very cost effective and my 8 port in my living room is fully capable of 100% utilization.
Here is a link: http://www.newegg.com/Product/Product.aspx?Item=N82E16833122364&Tpk=GS724T and http://www.newegg.com/Product/Product.aspx?Item=N82E16833122381&Tpk=GS108t

Full Duplex means full line speed ingress and full line speed egress at the same time. Not sequential between hosts.

Also Jumbo frames is HORRIBLE. Turn it off completely unless you are running iSCSI connections. PERIOD! Trust me on this. It will slow your network down.

Also linking (binding) teaming network adapters as in 802.3AD aka Link Agregation will not double your throughput speed between two hosts, but it will allow more bandwidth to be offered to when more than one hosts is pulling data from your device. I.e. 3 computers all pulling a file from your nas that has two nics in team mode will max at 2gbps to multiple hosts pulling from the nas but will not surpass 1gbos to any one host.
Click to expand...



3750E for a home network? geezes Time to step it up and place a used 6509 in your house with dual sups
 
/usr/home said:
If you want a decent Gig switch for a good price that can do LACP, check out a used HP 1800-24G or 1810-24G.
Click to expand...

+1, they are very affordable and do have a lifetime warranty on hardware. But since my new company is all cisco, I couldn't/wouldn't be able to switch back since the CLI is different.
 
You must log in or register to reply here.
Back
Top