Jumbo Frames ESXi 4,1

[jenuster]

I'm trying to setup jumbo frames ISCSI traffic on the VMkernel port and I noticed some weird behavior:

I am able to vmkping the management interface while specifying up to 65507 in size with the -d option. Why would I be even able to ping the the management interface with that size--especially if jumbo frames are not enabled.

Also, I am unable to ping the ISCSI interface with a jumbo frame deemed size even with the MTU's set to 9000 on the Vswitch1 & the vmknic.

Any ideas?


~ # esxcfg-vswitch -l
Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch0 128 3 128 1500 vmnic0

PortGroup Name VLAN ID Used Ports Uplinks
VM Network 0 0 vmnic0
Management Network 0 1 vmnic0

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch1 128 3 128 9000 vmnic1

PortGroup Name VLAN ID Used Ports Uplinks
ISCSI 0 1 vmnic1

# esxcfg-vmknic -l
Interface Port Group/DVPort IP Family IP Address Netmask Broadcast MAC Address MTU TSO MSS Enabled Type
vmk0 Management Network IPv4 192.168.1.101 255.255.255.0 192.168.1.255 00:25:90:34:79:6e 1500 65535 true STATIC
vmk1 ISCSI IPv4 192.168.1.111 255.255.255.0 192.168.1.255 00:50:56:7e:9d:d7 9000 65535 true STATIC

 

[NetJunkie]

You need to recreate the vmkernel interface with the MTU set to 9000. You can only set that when you create it.

 

[jenuster]

I tried that but still no luck.

 

[NetJunkie]

What happens if you ssh in to the vSphere host and ping via "vmkping -d -s 8500 ip.of.iscsi.target"?

 

[jenuster]

Ping to VMkernel interface
~ # vmkping -d -s 8005 192.168.1.201
PING 192.168.1.201 (192.168.1.201): 8005 data bytes
sendto() failed (Message too long)
sendto() failed (Message too long)
sendto() failed (Message too long)

Ping to Management Interface
~ # vmkping -d -s 8005 192.168.1.101
PING 192.168.1.101 (192.168.1.101): 8005 data bytes
8013 bytes from 192.168.1.101: icmp_seq=0 ttl=64 time=0.051 ms
8013 bytes from 192.168.1.101: icmp_seq=1 ttl=64 time=0.074 ms
8013 bytes from 192.168.1.101: icmp_seq=2 ttl=64 time=0.069 ms

Can't ping the iscsi target either.. Still troubleshooting (switch is enabled for JF and so is the Synology)

 

[NetJunkie]

Ah. You need to separate your vmkernel interfaces. If you have two on the same IP subnet it'll always use the first one. So everything is going to go out vmk0 which has an MTU of 1500. Just saw that on your OP. There is no way to force NAS traffic over a specific vmkernel port. I wish. Put iSCSI on a diff IP subnet.

 

[ND40oz]

Yup, I was just going to post that, it changed with ESXi 4.x, there's no real service console anymore, it uses vmkernel for everything and it'll use the first one available. Best practice to keep vmotion on a different subnet as well.

 

[jenuster]

Good call! Thanks

 

[defuseme2k]

Ah. You need to separate your vmkernel interfaces. If you have two on the same IP subnet it'll always use the first one. So everything is going to go out vmk0 which has an MTU of 1500. Just saw that on your OP. There is no way to force NAS traffic over a specific vmkernel port. I wish. Put iSCSI on a diff IP subnet.

I completely overlooked that, and probably wouldn't have seen it if you hadn't pointed it out. Reading comprehension 101.

I'm curious OP, why are you using jumbo frames? A learning excercise? If you're doing it to improve performance it probably isn't worth it as the results are a mixed bag. Jason Boche did an in-depth performance comparison (here). It might be worth a read for you.

 

[jenuster]

Yes, simply a learning exercise. I've read that blog post-- interesting results.

 

[lopoetve]

Yeah, from my experience, it's WAY more of a headache than a solution for 90% of what people do. Glad you got it working!