Jumbo Frames and vmkping

[vMikee386]

I have seen lots of sites that explain how to test jumbo frames by using...

vmkping -s 9000 {name/ip}

What I don't get, is why I can use 65,000 byte packets and have it work. I assume I am not actually sending 65,000 byte packets. So this tells me that the -s 9,000 is not proving I am sending 9,000 byte packets.

Can someone explain this to me?

# vmkping 10.28.109.13 -s 65000 -v

PING 10.28.109.13 (10.28.109.13): 65000 data bytes

65008 bytes from 10.28.109.13: icmp_seq=0 ttl=128 time=1.199 ms

65008 bytes from 10.28.109.13: icmp_seq=1 ttl=128 time=1.411 ms

65008 bytes from 10.28.109.13: icmp_seq=2 ttl=128 time=1.412 ms

--- 10.28.109.13 ping statistics ---

3 packets transmitted, 3 packets received, 0% packet loss

round-trip min/avg/max = 1.199/1.341/1.412 ms

#

[Josh26]

I have often wondered this.

In the Windows world at least, you've always been able to ping at any size, for example:

ping -l 50000 server

Would work fine. This is because your OS simply fragments the frames and lets it arrive in multiple frames.

To demonstrate jumbo frames, you would need to set the "do not fragment" but with the -f flag on your ping command. Only when doing this would you see oversize frames error out.

I never saw an equivalent switch in vmkping, and it therefore doesn't make a lot of sense to be testing jumbo frames this way, despite it being covered in many blog sites.

[vMikee386]

There is a -d switch, but I am not sure what it does. But if I invoke it, the anything over ~ 1490 bytes fails. This leads me to believe it has something to do with not fragmenting frames. That is if I am only working with standard frames and not jumbo ones.

[mittim12]

Could it have anything to do with the max MTU size you set on your devices? I know we don't get any response when we go above 9000 in our testing.

If you found this or any other post helpful please consider the use of the Helpful/Correct buttons to award points

Twitter: http://twitter.com/mittim12

[vMikee386]

To my knowledge, it is not possible to set MTU sizes above ~ 9000.

See the Wiki info here....[MTU|http://en.wikipedia.org/wiki/Maximum_transmission_unit]

-M

[vMikee386]

I've answered it myself. There is a switch (-d) for the vmkping command that invokes DF, which disables packet fragmentation. Is it possible this is new to 4.1?

# vmkping -s 8972 -d 10.28.109.13

PING 10.28.109.13 (10.28.109.13): 8972 data bytes

8980 bytes from 10.28.109.13: icmp_seq=0 ttl=128 time=0.379 ms

8980 bytes from 10.28.109.13: icmp_seq=1 ttl=128 time=0.380 ms

8980 bytes from 10.28.109.13: icmp_seq=2 ttl=128 time=0.387 ms

--- 10.28.109.13 ping statistics ---

3 packets transmitted, 3 packets received, 0% packet loss

round-trip min/avg/max = 0.379/0.382/0.387 ms

#

# vmkping -s 65000 -d 10.28.109.13

PING 10.28.109.13 (10.28.109.13): 65000 data bytes

sendto() failed (Message too long)

sendto() failed (Message too long)

sendto() failed (Message too long)

--- 10.28.109.13 ping statistics ---

3 packets transmitted, 0 packets received, 100% packet loss

#

[rickardnobel]

There is a -d switch, but I am not sure what it does. But if I invoke it, the anything over ~ 1490 bytes fails. This leads me to believe it has something to do with not fragmenting frames.

Interesting and very good that there is a -d switch for "do not fragment". (-f on Windows ping command). Without it any wmkping test is useless.

When it fails at anything over 1490 just means that the normal ethernet frame size is used, that is 1518 byte, including some overhead for ethernet and IP header.

[timpetro]

I know this is a long time closed, but the question you asked may be confusing two different issues.

1. the ping size specification is where you specify MTU size (MTU being the size of a single packet)

2. the 65535 mentioned in one of the questions most often refers to Window Size which is not the size of a single packet, but rather the amount of data you can send (in however many packets) before you must stop, wait for an acknolegement, then start transmitting another window of data.

I know if anyone has ever pingged with a 65535 size, which isnt possible because the largest jumbo frame is 9216; the ping packet is likely being fragmented into many small packets.

I hope if anyone was still wondering about this that this helps them.

[rickardnobel]

Tim Petrosky wrote:

1. the ping size specification is where you specify MTU size (MTU being the size of a single packet)

Hello Tim, I do not really think this is correct. The size mentioned in ping is not the MTU size, but the ICMP payload size. This is very important when testing if Jumbo Frames is correctly configured end-to-end. Since you do not specify the MTU you must subtract the IP + ICMP header sizes, or the test will fail. See this for some more information.

2. the 65535 mentioned in one of the questions most often refers to Window Size which is not the size of a single packet, but rather the amount of data you can send (in however many packets) before you must stop, wait for an acknolegement, then start transmitting another window of data.

The number of 65535 mentioned by someone above is really not the Window Size in this case, since that is something only found in TCP and not used by the ping commands. In this case 65535 means the maximum IP packet size which will, as you say, be fragmented by IP.

[timpetro]

Thanks for the reply; and perhaps you misunderstood my post, or i did not state it well. Let me clarify.

I was not implying that ICMP uses window sizes, and of course, they are only present in connection oriented TCP.
I was only suggesting that the original person who posted their question about trying to ping with a size of 65535 is likely confusing window size with MTU. Why? Because typically in IP that is where you see the 16 bit number, not in MTU. Then in their post they show it going thru, and of course its because its being fragmented because they did not include the '-d' switch in vmkping to prevent it.

Pinging and testing MTU on VMWare is quite easy to do and should be using sizes of 9000 - 8 for ICMP header - 20 for IP header.
Thus vmkping -d -s 8972 x.x.x.x is correct to test a giant.

Sorry if I didn't give a good explaination the first time.