How FC SAN traffic flows....

[MrVmware9423]

Dear Team,,

I just want to how FC SAn traffic flow.

for eg ESX Host hba1 is connected to SAN1 then what happs furter, how ESX Host HBA reach to it's destination....

Need basic and simple explanation on the same. or if sum1 have any gud video which explains the same then please share the link or video

regards

Mr VMware

[RParker]

SAN / FC is very simple.  The HBA is just the controller (works the same as your controller in your computer).  All the traffic goes over the Fiber cable, to a switch.  That switch is "zoned" to allow HBA to communicate with other HBA devices like the SAN.  The HBA has a path that is connected to a LUN on the SAN.  It's really no different than how IDE or SATA controllers work, you send requests for a hard drive data, and data flows over the appropriate cable to the hard drive, which is controlled by the driver.

I don't know what you mean "what happens further".  That's all part of the path from the ESX host to the Disk / Volume / LUN (destination)

[TedH256]

Wow - you don't want much, do you - just a complete tutorial on FC SAN storage communications!

lol!

At any rate, in simplest form, it is "follow the wire"

ESX is no different in this regard than any other FC connected host:

The storage array connects to the FC switches, and the hosts also connect to the FC switches. In the FC "fabric" (ie the backplane of the FC switches) the individual ports are ZONED so that the storage array and any connected host that needs to access storage, are allowed to communicate.

When a storage administrator "presents" a LUN to a particular host, that host uses FC protocol to see and then access the LUN.

Simple as that.

In most situations there are 2 separate FC "fabrics" and each storage array and host has (at least) 2 FC HBAs that connect - one to FC Switch "Fabric A", the other to FC Switch "Fabric B" - that way if an HBA or a FC switch port or even an entire FC switch are lost, the end devices (ie storage array and hosts) can still see each other.

HTH

--

Everything is vague to a degree you do not realize till you have tried

to make it precise.

Bertrand Russell

http://www.the-brights.net

http://xkcd.com/167

http://xkcd.com/836/

[RParker]

TedH256 wrote:

In most situations there are 2 separate FC "fabrics" and each storage array and host has (at least) 2 FC HBAs that connect - one to FC Switch "Fabric A", the other to FC Switch "Fabric B" - that way if an HBA or a FC switch port or even an entire FC switch are lost, the end devices (ie storage array and hosts) can still see each other.

HTH

I never setup my environments to be dual fabric, I made everyone one fabric.  The 2 switches (in my case) connected directly (dual fiber cable) for failover.. I don't see a purpose in having 2 separate fabric, especially since Brocade makes the switches connect in one huge fabric.. if it's a problem, why does Brocade allow switches to consolidate as one?

[TedH256]

Brocade makes the switches able to consolidate, because in a large environment, no single server could provide all the ports for any one fabric. You want separate fabrics so that in case anything happens to "the fabric" then there is another. Things that can go wrong with a fabric include more than just hardware failing - so really what you have done is probably fine, it just is a smidge less redundant - you are not protected from "fabric events" that could disrupt host access to storage ...

[EdWilts]

Brocade allows 2 switches to join the same fabric because you may exceed the ports in a single switch.

In our case, we have a director and an edge switch on 1 fabric - the edge switch is in a separate data center from the director.  This is fabric A.

We duplicate this for fabric B.  Every device that allows it (tape drives typically don't), we connect to both fabrics.

When you do firmware upgrades, you roll through one fabric at a time so that if there are fabric issues, you don't take out your complete environment.

It's all about risk management - you could run on a single switch with a single fibre cable and it will work.  Most of the time :smileyhappy:

[admin]

How ESXi access fiber channel storage?

How ESXi numbers a HBA?

How a VM accesses fiber channel storage?

[MrVmware9423]

Dear All,

Many thanks for the explanation.

On one the esx host 3  LUns are mapped from IBM XIV Storage ,  we are able to see only one path (no mutipathing ), then we have rescanned all FC HBA's of an ESX host, still same problem persist. then we have raise a concern with SAN team, they have informed us that ,from there end all is fine, requested us to check the same from vmware end, hence need ur help . from vmware side  what we have to do??? , so that we can see mutipathing on the same.      further SAN team suggested us to reboot ESX host to resolve the same. it's a production esx host , we won't be able to reboot the same.

Dear Prajapati,

many thanks for the snaps, if u have snap like this which explains VMware Infrasturcture, and help us to clear VMware Concepts how actually traffic flows  (SAN,ESX,Network,VM etc etc....., so plz help me with the same.its a  very self explanatory snaps,.if u have more snaps like this, request you to share  the same. or any URL where i can find snap like this.

regards

MR VMware

[a_p_]

Basically you should see "# of host ports" * "# of target ports" paths (e.g. 4 paths in case of 2 initiators and two target ports). If this is not the case start investigating by checking whether the host's HBA ports are healthy and show up on the fiber channel switch with their WWN with the correct speed. If this is the case ask the SAN team to check the zoning to ensure the host HBA ports are properly zoned to the storage ports. On the storage, the admins should have your hosts configured with all of the ESXi host's HBA WWNs.

How many paths do you see on the other ESXi hosts?

André

[MrVmware9423]

Dear Andre,

we are able to see esx hba wwpn is logged in on a SAN switch and following is the zone configuration details

SAN1 configuration

ESX HBA1 WWPN

IBM XIV SP1

IBM XIV SP2

SAN2 configuration

ESX HBA2 WWPN

IBM XIV SP3

IBM XIV SP4

we are only able to see one path i.e. IBM XIV SP1.

request u to let me know if any other info is required from my end.

regards

Mr VMware

[a_p_]

I'm not familiar with this specific IBM storage system. However, if your ports show up on the FC switch, zoning is configured properly (unless someone forgot to enable the active configuration after changing it) and other hosts can see the paths to the other storage ports, then I cannot think of another place to look at than the storage configuration (maybe a typo in the WWN). Has this setup worked before or has the host been added to the environment just recently?

André

[MrVmware9423]

Dear Andre,

Just want to ask 1 thing, ocne we inform SAN team about mutipathing issue then these ppl done the needful changes on SAN switch and informed us like that every thing is fine, then is it require to reboot host to take effect. Mutipathing is depend on SAN switch configuration ???

regards

Mr VMware

[a_p_]

No, a host reboot should not be required, because there are no changes done to the Hypervisor. Rescanning the HBAs should be sufficient to detect the new paths to the storage/LUNs.

André