Ok, so I started playing around, creating a second gateway on NSX and presenting it as a second provider+edge gateways on VCD (just like I did with the first one) but this time I used IP blocks instead of IP spaces and noticed a LOT of differences. I also gave my orgadmin all the related rights for Provider Gateway management inside the Organization.
Private Provider gateway using IP Spaces:
- Provider Gateway tab is available on the Networking tab inside the Organization;
- Provider Gateway has the following BGP options available: CONFIGURATION (Status, AS Number, Graceful Restart options, Stale Router Timer and ECMP), NEIGHBORS, IP PREFIX LIST, COMMUNITY LISTS, ROUTE MAPS, PERMISSION GROUPS);
- Able to modify all BGP options EXCEPT the ones in the CONFIGURATION tab (so orgadmin is unable to disable BGP or change AS number, for example);
- Related Edge Gateway has NO BGP configuration options at all;
- Related Edge Gateway able to create static routes;
- Related Edge has Route Advertisement as read only (unable to change it);
- Configuration availability is consistent across Org view and System view;
- Behaviour is consistent with either T0 or VRF backed Provider Gateway.
Provider Gateway using IP Blocks (set as dedicated when creating Edge Gateway):
- Provider Gateway tab is available on the Networking tab inside the Organization;
- Provider Gateway has NO configuration AT ALL (absolutely nothing can be configured);
- Related Edge Gateway has the following BGP options available: CONFIGURATION (Status, AS Number, Graceful Restart options, Stale Router Timer and ECMP), NEIGHBORS and IP PREFIX LIST);
- Related Edge Gateway is able to modify ALL BGP options if TO (if VRF on CONFIGURATION only able to enable/disable BGP and ECMP);
- Related Edge Gateway able to create static routes;
- Related Edge has full Route Advertisement control (enable/disable and define subnets);
- Configuration availability is consistent across Org view and System view;
- Behaviour is consistent with either T0 or VRF backed Provider Gateway (except for BGP CONFIGURATION as mentioned above).
Now I understood why my API calls were talking about invalid object.
With that I can see there are some inconsistencies with the official documentation:
- Here mentions that "If you are using NSX 4.1, you can edit the the local AS number on an edge gateway that is backed by a VRF gateway" but that is not true;
- This other link mentions that "If you are using a dedicated provider gateway with IP spaces, your system administrator configures static routes and BGP on the provider gateway, and you can manage BGP settings for your NSX edge gateway that is backed by the dedicated provider gateway. You configure route advertisement on the organization VDC network level" but as I mentioned I'm only able to do use with IP Blocks.
There are those as examples and also the fact that it actually took me a while to understand that PRIVATE (IP Spaces) Provider Gateway and DEDICATED (IP Blocks) Provider Gateway are two completely different things.
In the end its my decision on which options I'm gonna make available to my customers but sadly they won't get the full set :-/