Can't add static routes with netmgr

[itsjacob]

I just installed fresh Cloud Director 10.3.3 appliance cells (running on Photon 2.0). I discovered that I was missing a static route, and tried to add it - but I couldn't get any variation of netmgr ip_route --add to work. It keeps returning "Error: invalid parameter" for anything that I supply as a destination. I tried single and double quotes, with and without a mask prefix, a different network address... same results.

I was able to work around it by manually adding the routes to the config files at /etc/systemd/network, but I don't know why netmgr isn't working for me. netmgr is v1.1.0, and tdnf doesn't find any updates for it.

Example workflow:

 

root@vcdcell [ ~ ]# netmgr ip4_address --get --interface eth1
IPv4 Address Mode: static
IPv4 Address=10.26.231.135/24
IPv4 Gateway=10.26.231.1
root@vcdcell [ ~ ]# netmgr ip_route --get --interface eth1
Static IP Routes:

Error: value not found
root@vcdcell [ ~ ]# netmgr ip_route --add --interface eth1
Usage:
ip_route --get --interface <ifame>
ip_route --add --interface <ifname> --gateway <GatewayIP> --destination <DestinationNetwork/prefix> --metric <N>
ip_route --del --interface <ifname> --destination <DestIP/N>
root@vcdcell [ ~ ]# netmgr ip_route --add --interface eth1 --gateway 10.26.231.1 --destination 10.0.0.0/8 --metric 1
Error: invalid parameter
root@vcdcell [ ~ ]#

 

 

[Daniel Casota]

Hi,

I'm not familiar in depth with the commercial vCloud Directory appliance' dedicated cli tools which encompasse os level and cell higher level configurations. This is certainly important as commercial appliances with Photon OS are not open-source Photon OS.

In theory there are multiple ip configuration possibilities:

- ovf properties (provisioning phase only)
- vCloud Directory dedicated configuration tool
- netmgr (in theory, it could be a vCloud Director specific version netmgr 1.10)
- systemd network file(s) and resolv.conf

Accordingly to the known issues in vCloud Directory 10.3.3 release notes, adding a ipv4 nic fails in a certain situation.

For the open source Photon OS 2.0, the netmgr issue "Error: value not found" is reproducible.

Systemd allows to configure configuration network file(s) and nameservers in resolv.conf, see attached notes.txt. Be aware, as said, this was in a testlab,  tdnf update and any additional package installation (iputils) is NOT recommended in prod environment because it can mutual damage the appliance so that update/upgrade scenarios will fail.

For vCloud Director, also see https://www.reddit.com/r/vmware/comments/k065xi/changing_eth1_ip_vcloud_director_appliance .

This works flawlessly in the testlab:

netmgr ip4_address --set --interface eth0 --mode static --addr 192.168.0.205/24 --gateway 192.168.0.1

ip route allows to configure additional routing.

Hope it helps.

 

Attachment(s)

notes.txt   956 B 1 version

[itsjacob]

I didn't have any problems editing the IPv4 address - it was only the command to add static routes that was failing.

I can ask around on the vCloud Director side, but I figured I'd start here since it looked like a netmgr issue. Didn't realize that there was that much potential for difference between Photon bundled with VMware appliances and the official releases.

[Daniel Casota]

Unfortunately, with the latest Cloud Director 10.4.1, which includes support for vSphere 8.0 ++, the issue mentioned still isn't fixed.
Well, for in a production environment, paying for a feature/hotfix is the Executive way.

edited December 15th 2022:
Photon OS as router, see the VMware Tanzu homelab example from  . In theory unbound and bindutils should work on Photon OS 2.0 as well.

Accordingly to the VMware product lifecycle matrix, Cloud Director 10.3 reaches End Of General Support at 2023-07-15. That said, opening an SR based on Support&Subscription, is a good way of getting help and an official statement about work-around(s) or fix(es), too.

Accordingly to the announcement by Kishan Malur, Product Line Manager with Cloud Platform Business Unit, in January 2022, the open-source Photon OS 2.0 reaches End of Support. That said, placing a new issue request at https://github.com/vmware/photon could end as "won't be fixed" for the open-source version of netmgmt.

Netmgmt is a supported VMware package. The source in Photon OS 2.0 however wasn't maintained since 2017.

Meanwhile the demand to control os services using Rest API increased, and the Photon OS project team decided not to fusion netmgmt + Rest API capability + management daemon. It was too clunky to maintain it for all flavors, x86_64 and arm64, and for AWS/GCE/Azure, etc. Inspite of that, they implemented Photon Management Daemon Next Generation. Pmd-nextgen was officially announced for Photon 4 rev 2. Simply compare the content in the docs pmd-nextgen and netmgmt. This is huge because it can be easily adopted for Telco automation with K8s Worker Nodes. The design implemented is superior because it has this nifty plugin thinggy for dynamic onboarding/offboarding/mutations of connections.

Imho the best part is the Photon OS project team because the people there know each other for light years and they can help other teams to unleash new appliance releases with latest update/upgrade/side-by-side bits. Developers know things.

[Daniel Casota]

Double checked the assumption

https://github.com/vmware/photon/issues/1382

 

[itsjacob]

That's a bummer, but at least we know now. Thanks for taking the time to ask around and look into this!