Milan,
It is probably going to be simpler to setup a fresh environment under the new hostnames against a fresh database and then import your Process Definitions. Once that environment is up and functional simply change the pointers in the F5 pools to point to the new PAM servers.
Process Automation depends on the hostname, changing the hostname will have an impact on your running instances. Each instance is set to run on a UUID that equals server 'pam1.domain.corp' if you change the names to 'yyy.newdomain.corp' none of those existing running instances will be able to resume functioning as the UUID no longer exists.
You could change your load balancer without issue as long as it redirects to the Process Automation hostnames correctly, but you will need to run the installer to point PAM at the new name for the load balancer.
"Is it any way how to rename balancer name and PAM nodes without re-installation?"
No, you will have to run the installer, it is required to change from http to https as the installer has to apply the Self Signed certificate. During the installer you could modify the load balancer name, and Process Auto hostnames at the same time you apply the SSL certificate.
After renaming we need to enable Secure Communication. Is it supported wildcard certificate for Secure Communication?
I cannot find any details for or against a wild card certificate. General google searches do not turn up any issues with wild card certificates in JBOSS the underlying application server for Process Automation.
You can get more details on the certificate usage in Process Automation here:
Manage Certificates - CA Process Automation - 4.3 - CA Technologies Documentation
Finally, You could follow this tech doc:
How do I configure a new PAM Orchestrator instance to use existing PAM database tables?
which would preserve your instance history, but again, in flight, or running, instances will not function after this process.
If you are not setting up a fresh environment under the new hostnames and would like to attempt to rename the existing environment, I urge caution and thorough testing prior to attempting this in Production.