FT :Packet drops during failover

[helltejas]

• Hi,
  

• ->I enabled FT in one VM where we created 1 C++ program to receive UDP packets
  
• ->I am sending this UDP packet from my PC to FT VM in 10 ms time interval
  
• ->I am using Jumbo frames and it is enabled also on Switch
  
• ->VM has XP OS and it is using Flexible vNIC
  

• issue:
  
•   I am facing 91 packet drops
  

• changed config in Advance options:
  
• FT.Heartbeatcount    10 to 5
  
• FT.hostTimeout 2000->100
  
• FT.LogTimeout  8000->500 ms
  
• FT.Backupconnectimeout  8000->100 ms
  
• FT.TCPSendFuffSize  131072 -> 16384
  
• FT.TCPRcvBufSie   131072->16384
  
• FT.PrimaryConnectTimeout   8000 -> 100
  

• After Change:
  
•   packet drops were reduced 91 packet drops to 41
  

• We want to achieve 0 packet drops
  

• Any suggestion?
  

[AmeenMunaff]

Make sure your FT logging network is seperate from vMotion network.

[helltejas]

thanx 4 quick reply,

Do i have to create seprate VLAN or only different subnet will work fine?:smileyconfused:

[AmeenMunaff]

Try Both, in my experience i have kept vmotion & FT with different subnet. . but as you wanted 0 drops, try creating a seperate port group :smileywink:

if different subnet duznt help then only go for seperate port group!

[helltejas]

I created 1 FT and 1 VMotion Portgroup in two different vSwitchs.

vSwitch is connected to 10G pNIC with jumbo frame enable

still not working

NOTE: vMotion and FT subnets are in same subnet

if i change subnet will it solve my problem?

[AmeenMunaff]

it should!! Please try and let me know...

in the mean time will put my brain to work :smileylaugh:

[iw123]

Does this only happen when the primary VM has a failure or during normal operation?

[helltejas]

It happens during failover and when secondary vm (shadow VM) was being created.

Normally it works fine 0% packet loss

[AmeenMunaff]

whats the nic you have? 1 gig or 10 gig.. and how many FT vms you have in a host.

[helltejas]

10 gig nic in server and switch

[helltejas]

Both host have 10gig ports connected to 10gig switch ports

jumbo frams enabled on switch as well as on esxi vswitch and ft port group

FT is enabled in 2 VM

it drops packets when it do a failover and during the time it creates secondary vm

i also tried to change subents and vlans but still 41 packet drops...:smileyconfused:

[Linjo]

Not an expert on this but I think that some packetloss is expected during the short stun that will take place during the failover.

With UDP (since its not reliable) it should be assumed that not all packet will reach the destination, if you need guaranteed delivery then you need to use TCP that has resend functionality builtin.

Do you see the same behavior while taking a live snapshot? (I would excect similar behaviour since there also will be a short stun)

// Linjo

[helltejas]

After changing FT.Backupconnectimeout  8000->100 ms, packet loos reduced to 41

100 ms is the minimum value we can set here.

so is this a vmware limitation in FT?

then why VMware says that FT gives 0 packet loss:smileyconfused:

[Linjo]

Where are they saying that? (From my point of view its a hard thing for a software vendor to guarantee UDP packet delivery, but as I said earlier this is not my field so I might be wrong.)

// Linjo

[AmeenMunaff]

How about MTU value for FT & vMotion port group? can you try setting it to 9000 and check

[helltejas]

Hi Ameen,

    MTU is already set to 9000 on all vSwitch all Portgroups and on all Physical switch ports.

FT has 10 gig port where other vSwith has 1 gig ports.

[helltejas]

1 ping packet is also getting drop...

dose anyone faced this performance issue?

[Linjo]

Had a look at the messagaging around FT and what I could find is that we will not drop any TCP traffic during a failover, that is very different from dropping ICMP and UDP packets.

So from my point this is not a performance issue and it works as designed and the failovertime is very low.

If you feel this is an issue you should contact VMware Support and see what they say about it.

// Linjo