I believe you should make sure that your different subnets have the reachability to each other.
Did you tried to ping and telnet the ESXI host from 50.X subnet machine.
same subnet is reachable reason a visibility in same subnet . This should be a problem in your network configuration in you design.