VMware GemFire

 View Only

 Can Gateway sender send to Gateway recevier staying in same group?

Futures Support's profile image
Futures Support posted Jan 05, 2021 12:39 PM

Hi Support,

We recently enable WAN replication on our gemfire cluter cross data center. Each cluster contains 3 locators and multiple groups of gemfire servers. Servers contains diff regions in diff group.

 

When test we find that gateway sender connects to receivers randomly, e.g. sender on servers belong to group A sends data to receivers belongs to group B, causing that region not found when receiver processing the data. Below is exception. Is there any way to restrict sender connects to receivers in same group?

 

[ServerConnection on port 1571 Thread 130 : 2021-01-05 03:14:31,885] BaseCommand WARN - Server connection from [identity(169.185.205.144(PrimaryGemfireServerXF:32323)<v7>:41002,connection=1; port=38228]: Wrote batch exception: 

org.apache.geode.internal.cache.wan.BatchException70: Exception occurred while processing a batch on the receiver running on DistributedSystem with Id: 106, DistributedMember on which the receiver is running: 10.210.60.42(PrimaryGemfireServerC:59287)<v2>:41001

   ...

Caused by: org.apache.geode.cache.RegionDestroyedException: Region /root/fusion/tmg/envelopeObject was not found during batch create request 2709

    at org.apache.geode.internal.cache.tier.sockets.command.GatewayReceiverCommand.handleRegionNull(GatewayReceiverCommand.java:77) ~[geode-core-9.9.1.jar:?]

    at org.apache.geode.internal.cache.tier.sockets.command.GatewayReceiverCommand.cmdExecute(GatewayReceiverCommand.java:404) ~[geode-core-9.9.1.jar:?]

    ... 9 more

 

Juan Ramos's profile image
Juan Ramos

Hello @Futures Support​ ,

 

Thanks for contacting the Support Community!.

At the top of my head, I don't believe this is possible at the moment, every gateway-receiver assumes that the target region on which to apply the received event actually exists locally, which is not the case here as the regions are actually split into server groups.

I'd suggest to open a Support Ticket stating the use case and business requirements, however, and request the Support Engineer for the details and/or process about how to open a Feature Request with the GemFire Product Management Team, as this might be a nice feature to have.

Best regards.

Futures Support's profile image
Futures Support

Hi Ramos,

Then you mean the only way to solve this is to split server groups into diff cluster? Or do you have better suggestions?

Futures Support's profile image
Futures Support

Or is there any way to make sender connects to specific member/receiver rather than determined by locator randomly?

Juan Ramos's profile image
Juan Ramos

Hello @Futures Support​,

 

Splitting the servers from different groups into different clusters instead should work, but it implies an extra burden related to management and deployment on your side, that's why I didn't even suggested that option. Also, there's no way of configuring a gateway-sender to connect to a specific gateway-receiver.

The documentation here [1] states that the same regions should be configured both in the source and target clusters in order for the distribution to work properly, but it doesn't say anything about supporting or not the usage of server groups.

As a said before, this would be a nice feature to have in the product, so I'd suggest to open a Support Ticket and ask for the steps to create a Feature Request.

Best regards.

 

[1]:https://gemfire.docs.pivotal.io/910/geode/topologies_and_comm/multi_site_configuration/setting_up_a_multisite_system.html

Futures Support's profile image
Futures Support

Hi Ramos,

 

"the same regions should be configured both in the source and target clusters in order for the distribution to work properly" -- regarding to this yes the regions on both clusters are exact same. It is regions on members are diff in same cluster as we split as groups like below,

cluser 1:

groupA -- member 1 - region X

groupB -- member 2 - region Y

 

cluser 2:

groupA -- member 1 - region X

groupB -- member 2 - region Y

 

when replication data was sent from member 1 on cluster 1 a to member 2 on cluster 2. Our expectation is sending data to member 1 on cluster 2. I think this should be an usual case that we cannot count on all regions are same in a cluster right? If so then why we introduce concepts of server group?

Juan Ramos's profile image
Juan Ramos

Hello @Futures Support​ ,

 

I agree with you in that this use case should be supported but I don't see this scenario being handled within the source code, so it seems to me this is an undocumented limitation related to WAN Topologies, that's why I'm suggesting to open a Support Ticket directly.

Best regards.

Futures Support's profile image
Futures Support

Hi Ramos,

 

How to open a ticket? could you help on that?

Juan Ramos's profile image
Juan Ramos

Sure, you can find all the information in the following link: https://tanzu.vmware.com/content/support/support-onboarding.