Im working with IBM and VMware to try and figure it out. Here is what IBM says:
The only errors we are seeing in the logs for the storage are "mode sel page 2 c" messages which is an indication of "Lun Flapping".
Lun Flapping is a term we use when a host has multipathing configured but the host is trying to send traffic from one
hba port and then from the other hba port to the same logical drives. Lun Flapping is caused by poor configuration.
It results in the write cache on the controller being disabled until one path is settled on....and that causes latency on the
hosts.
The fix for Lun Flapping will be with software Configuration Support.
Please have the VMWare admin refer the fail over policy in VMWare. The policy should be set to Most Recently Used (MRU) and the WWPN should match the WWPN of Controller A.
Here is what VMware says so far:
The messsages coming from the Storage Views are directly connected to the storage array. If you are receiving messages you should contact your vendor to make sure everything is ok.
We are using MRU but I assume the WWPN doesnt apply since we are using iSCSI