I was going to give up on this, tried official guide https://kb.vmware.com/s/article/2148128 and many other guides and still no joy, it still complained "Failed to find key file".
Found another guide at https://harrythambi.com/posts/vrni/2022-08-14-replacing-certificates-for-vmware-vrealize-network-insight/ , it seems the difference is that there is a step to “fix” the headers in the key file, by running the following commands:
sed -i 's/-----BEGIN PRIVATE KEY-----/-----BEGIN RSA PRIVATE KEY-----/g' vrni_key.key
sed -i 's/-----END PRIVATE KEY-----/-----END RSA PRIVATE KEY-----/g' vrni_key.key
Before that fix , when I tested the key file with "file vrni_key.key" it was showing "ASCII text", only after fixing headers it showed "PEM RSA private key".
There seems to be a bug in certain versions of OpenSSL where the file type of an RSA Private key shows to be "ASCII text", funny thing is that even when you generate the key in VRNI 6.10.0.1680883302 it still has this bug and the headers need to be fixed for the key to be valid.