SRM REST API v2 'configure-all-vms' returns "HTTP 404" when protection group contains VM with status NOT_CONFIGURED

[ririmia]

Hello,

I'm trying to use SRM API v2 to protect VMs from a Protection group.

https://developer.broadcom.com/xapis/vmware-site-recovery-manager-rest-api-gateway/2.0.0/pairings/pairing_id/protection-management/groups/group_id/actions/configure-all-vms/post/

'configure-all-vms' has a strange behaviour.

If all VMs are protected the API call response is 202 Accepted with Response body: {"list": [], "_meta": { "total": 0 } }
If at least one VM is unprotected the endpoint response is 404 Not Found with no useful response body.

More information:

VMware Site Recovery Manager v 9.0.2, 24639340 // VMware Live Site Recovery REST API v2 // Array Based Replication
Any other SRM API endpoint is working. Authentication is working. I'm getting the proper session id. Any other GET\POST on other endpoints are working without any issues.

Is 'configure-all-vms' supported for ABR protection groups in this SRM version?
Is the observed 404 behavior expected when a VM is in NOT_CONFIGURED state?
What REST API operation should be used to re-protect a VM after protection has been removed?

Thanks!

[Allan Solomon Mejia]

Based on that, receiving a 404 Not Found only when the group contains a NOT_CONFIGURED VM doesn't look like expected behavior. The request is reaching SRM correctly because the same endpoint returns 202 Accepted when every VM is already protected. This points more to an SRM REST API issue in 9.0.2 than an invalid URL, an authentication problem, or an unsupported ABR operation.

 

There have also been earlier SRM releases where VM protection REST operations incorrectly returned HTTP 404 responses, so I'd recommend:

 

• Confirm that Configure All VMs works for the same protection group through the SRM user interface.
• Review /var/log/vmware/srm/vmware-dr.log immediately after reproducing the API call. The server-side log should contain the underlying exception even though the REST response body is empty.
• Test against the latest supported 9.0.x maintenance release, if possible.
• If the issue remains, open a Broadcom support case with the request URL, operation ID or timestamp, SRM logs, and the affected protection-group and VM identifiers.

 

As a possible API workaround, retrieve the affected VM and its current protection settings, then update the individual VM through the Get VM Protection Settings and Update VM Protection Settings operations. Use the object returned by the GET request as the basis for the PUT request, changing only fields marked editable.

 

For an ABR protection group, a VM on a replicated datastore is automatically discovered as a group member. After its protection is removed and its state becomes NOT_CONFIGURED, configure-all-vms should be the correct bulk operation for configuring it again. A 404 in that state should therefore be treated as unexpected behavior.

[ririmia]

Hi Allan,

Thank you for your reply.

• Confirm that Configure All VMs works for the same protection group through the SRM user interface.

Everything working as expected from SRM user interface. All the mappings are correctly configured. I have issues just with SRM API.

• Review /var/log/vmware/srm/vmware-dr.log immediately after reproducing the API call. The server-side log should contain the underlying exception even though the REST response body is empty.

Great information. I didn't knew that SRM API endpoint logs are in that file. I'll check.

• Test against the latest supported 9.0.x maintenance release, if possible.

This is a real issue as I need to comply with storage vendor comaptibility matrix (SRA) but I'll do again a review if something new was release by stoarge vendor.

• If the issue remains, open a Broadcom support case with the request URL, operation ID or timestamp, SRM logs, and the affected protection-group and VM identifiers.

Already open the SR case but lately the quality of support is getting lower and lower day by day.
This is Broadcom SR answer:
-----------------------------------------

> Is the observed 404 behavior expected when a VM is in NOT_CONFIGURED state? Yes 

> Is 'configure-all-vms' supported for ABR protection groups in this SRM version?

All VMs within the targeted protection group should be in a healthy, "OK" status for the configuration operation to process successfully. If even one VM in the group is in an unconfigured state, the API call will fail with the 404 error.

> What REST API operation should be used to re-protect a VM after protection has been removed?

To re-protect a virtual machine please ensuring the VM is in a valid state

Verify that all required network, folder, and resource pool mappings are correctly configured.

If VM protection state has become corrupted or invalid, please resolve it through the UI to restore an "OK" status.

-------------------------

Based on this answer I need to make sure that VM is already protected to be able to use SRM API to protect the VM.
Does it make any sense?

[ririmia]

Thank you, Allan!

 

Your suggestion pointed me in the right direction.

 

After checking the logs, I found the following error immediately after making the API call:

 

"026-09-18T09:06:54.034+02:00 verbose vmware-dr[1298640] [SRM@6876 sub=authorize.EntityResolver opID=c951959f-0c49-4821-a3ce-8e739503967b-protectVms] [52556] Failed to resolve dr.replication.VmProtectionGroup:93c54391-1f69-4f5b-afa0-1c034f5e193c:vm-protection-group-52442900"

 

It appears that when at least one VM in a Protection Group is not protected, the SRM server on the DR site does not contain that Protection Group in its database. At least, that is my current assumption, since the error occurred only when I was using the SRM API endpoint from the DR site.

 

When I use the SRM server from the protected site, the configure-all-vms API call works as expected.

 

In the end, the issue was caused by me accidentally using the wrong SRM API server. The SRM UI displays all Protection Groups regardless of which site protects them, which made it easy to overlook this distinction.

 

Thanks again for your help!