• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Can I Bind Two NIC's Together

C

Carlosinfl

Loves the juice
Joined
Sep 25, 2002
Messages
6,633
I have a blade enclosure with 5 blades. Every blade is running VMware ESXi v5. Currently we have just one CAT6 cable from the blade to the network switch and my question is can I bind / bond / trunk / combine the 2nd NIC on the blade to double the bandwidth? I would like to combine the two 1000 Mbps per blade to have a larger pipe to pass data to and from the switch. Is this possible or supported on ESXi and if so, can someone please tell me what all would be necessary to complete this task?

Thank you!
 
LACP mode is not supported for use with ESXi with vSS and vDS, however I do believe it's supported with the Nexus 1kv, i'll defer to Netjunkie, however.
 
Vader said:
LACP mode is not supported for use with ESXi with vSS and vDS, however I do believe it's supported with the Nexus 1kv, i'll defer to Netjunkie, however.
Click to expand...

Correct. LACP isn't supported on either integrated vSphere switch but is supported by the N1Kv. The mode you use on the switch side depends on which hashing mechanism you use. If you use the standard "Hash by Virtual Port ID" then you don't do anything on the switch for channeling since it's basically done via MAC address (Same with Hash by Source MAC). Easy and works well for most people.
 
ok, I'll dive in on the end as my question is kind of relevant.

I am looking at setting up a home esxi installation
Dell R210 for VM Box, with 2 x LACP bonded nics (2 onboard GigE + Dual GigE with TOE / iSCSI) to switch (2GB data, 2GB iscsi to Openfiler) is that a viable config or am I looking trying to achieve the impossible. The iSCSI will run in a seperate vlan as well

from what I have read its perfectly possible (The switch supports it)
 
You can bond NICs..you just don't do it with LACP. vSphere doesn't do LACP without the Nexus 1000v so you hard set the bonding configuration on both sides.
 
I thought that it didn't do automated MPIO, but does natively do failover or round robin which can be tweaked.
 
Pardon my possible ignorance as I'm new to the whole virtualized world. So I'm not able to have the below setup:

ESXi ---> 2x1GB links for data ----> LACP enabled ports on my switch --->2x1GB links for data ----> FreeNAS box with the 2 prev. ports aggregated

If so, that sticks. I overbought some hardware then.
 
peanuthead said:
Pardon my possible ignorance as I'm new to the whole virtualized world. So I'm not able to have the below setup:

ESXi ---> 2x1GB links for data ----> LACP enabled ports on my switch --->2x1GB links for data ----> FreeNAS box with the 2 prev. ports aggregated

If so, that sticks. I overbought some hardware then.
Click to expand...

FreeNAS: Bind NIC and use LACP
ESXI: Use two nic and setup round-robin

Round-robin will still only use 1 of X link setup but if you have multiple VM and multiple esxi cluster then you will be able to saturate your xGb link. I have a Freenas with 2x1Gb and 2 esxi cluster and i get near 2Gb of traffic out of my FreeNas server.

At worst go with 4Gb Fiber or 10Gb, multiple link is a redundancy at the same time not only about speed.
 
Thanks, Altivec. I agree, but naturally I would like more bandwidth. Who wouldn't? :)

I think I should be good to go then. I may have 2 extra pci-e NICs but I can always resell them. Just so I can validate my proposed setup; are there are good and simple to read links on VMWare link configurations?
 
Yes but do you need 2Gb/sec to a single VM ?

If you really need such bandwidth to one VM then the expense for a 10Gb Ethernet or 4Gb Fiber won't be a problem.
 
I don't think we're talking about 2Gb of network traffic to a VM, are we? We're talking really about 2Gb to storage.
 
peanuthead said:
Pardon my possible ignorance as I'm new to the whole virtualized world. So I'm not able to have the below setup:

ESXi ---> 2x1GB links for data ----> LACP enabled ports on my switch --->2x1GB links for data ----> FreeNAS box with the 2 prev. ports aggregated

If so, that sticks. I overbought some hardware then.
Click to expand...

Won't work from the ESX side unless you have Nexus 1k, but even then, you won't get what you expect.

altivec said:
FreeNAS: Bind NIC and use LACP
ESXI: Use two nic and setup round-robin

Round-robin will still only use 1 of X link setup but if you have multiple VM and multiple esxi cluster then you will be able to saturate your xGb link. I have a Freenas with 2x1Gb and 2 esxi cluster and i get near 2Gb of traffic out of my FreeNas server.

At worst go with 4Gb Fiber or 10Gb, multiple link is a redundancy at the same time not only about speed.
Click to expand...


peanuthead said:
Thanks, Altivec. I agree, but naturally I would like more bandwidth. Who wouldn't? :)

I think I should be good to go then. I may have 2 extra pci-e NICs but I can always resell them. Just so I can validate my proposed setup; are there are good and simple to read links on VMWare link configurations?
Click to expand...

altivec said:
Yes but do you need 2Gb/sec to a single VM ?

If you really need such bandwidth to one VM then the expense for a 10Gb Ethernet or 4Gb Fiber won't be a problem.
Click to expand...

NetJunkie said:
I don't think we're talking about 2Gb of network traffic to a VM, are we? We're talking really about 2Gb to storage.
Click to expand...

I have said this a million times, and I know I'll say it a million more. This is the mistake that every customer makes at first - when I present storage practices, this is the top topic every single time.

You NEVER run out of interconnect. You always run out of IOPS. 1GE, 4GFCE, pfft - none of it matters. You run out of disk WAY before you run out of link - remember, you've got multiple VMs and multiple hosts, so data to the array is always 100% random, completely different sizes, and will kill IOPS before you come close to killing the link.

A MD3000 can run rings around a symmetrix or CX4 if you have the disks set up wrong, regardless of link speed (discounting cache, of course, but that's the same thing - cache before link).

Worry about the disks, not the links.
 
What server blades are we talking about. What are the interconnects at the back of the blade chassis?

Server - ServerNic1 -Blade SwitchA - Physical Switch
- ServerNic2 -?
 
Sorry to keep jumping topics...but I am just as confused.

So I just set up up two gigE links in MPIO round robin from ESXi 5.0 U1 to my synology DS1512+.

Aside from redundancy, what did I gain over a single NIC configuration? Over two gigE NICs without MPIO?
 
RezQme said:
Sorry to keep jumping topics...but I am just as confused.

So I just set up up two gigE links in MPIO round robin from ESXi 5.0 U1 to my synology DS1512+.

Aside from redundancy, what did I gain over a single NIC configuration? Over two gigE NICs without MPIO?
Click to expand...

For a single host with a single lun? Nada
For a single host with multiple luns? Each lun's pathing is unique, so each one (with RR) could use either link at any time (rotating every 1000 commands), and thus eliminate port contention if it exists.
For multiple hosts with multiple luns? See item 2, only now it's per host - eliminates most common constraints with the links (buffer on receive/send). But you run out of disk first still.
 
RezQme said:
Sorry to keep jumping topics...but I am just as confused.

So I just set up up two gigE links in MPIO round robin from ESXi 5.0 U1 to my synology DS1512+.

Aside from redundancy, what did I gain over a single NIC configuration? Over two gigE NICs without MPIO?
Click to expand...

What Iopoteve said...

You're adding another path and the bandwidth that comes with it for the Host to use on a PER connection basis. However, since almost all Disk traffic is random, you'll still probably be limited by the overall random IOPS being asked of the physical disks before you'll max out bandwidth. Think of it as adding another register at the McDonalds counter. Now you can serve twice as many people, but only at the speed of the Kitchen.

There's ways to Tweak VMWare's Round Robin multipathing, but the bottleneck will always be the physical disks, unless they're SSD's.
 
Last edited:
Nate7311 said:
What Iopoteve said...

You're adding another path and the bandwidth that comes with it for the Host to use on a PER connection basis. However, since almost all Disk traffic is random, you'll still probably be limited by the overall random IOPS being asked of the physical disks before you'll max out bandwidth. Think of it as adding another register at the McDonalds counter. Now you can serve twice as many people, but only at the speed of the Kitchen.

There's ways to Tweak VMWare's Round Robin multipathing, but the bottleneck will always be the physical disks, unless they're SSD's.
Click to expand...

And even then, given how we abuse the shit out of disks, most MLC disks will get NASTY capped by the garbage collect logic. Don't expect 30k+ iops :D much better than normal disks, mind you, but not magic :D
 
Thanks for the info, guys.
I currently have a single host with a single LUN.... :-\
 
You must log in or register to reply here.
Back
Top