vSphere Storage Appliance

 View Only
Expand all | Collapse all

Configuring iSCSI dependent hardware

  • 1.  Configuring iSCSI dependent hardware

    Posted Mar 18, 2011 09:20 PM

    Hi to everybody I've a small question...

    My system is now configured with an iSCSI software adapter in this way:

    - Virtual switch called iSCSI with a VMKernel port for the iSCSI with two NICs in teaming.
    - iSCSI software adapter

    Now I want to use my two new iSCSI broadcom dependent hardware adpater.

    Following the iSCSI configuration guide official by VMware I must to configure the system in this way but I have
    perplexity about it.


    - two VMkernel Ports in same subnet (one for each iSCSI adapter).
    - each VMkernel port linked to it phy_NIC
    - both VMkernel ports bound to the relative iSCSI hardware adapter.


    Is that right?
    Can I have two VMKernel port in the same subnet?

    Thanks in advance for your advice!



  • 2.  RE: Configuring iSCSI dependent hardware

    Posted Mar 18, 2011 11:42 PM

    Hi moneta82,

    http://www.vmware.com/pdf/vsphere4/r41/vsp_41_iscsi_san_cfg.pdf explains the necessary steps in a good way, but basically you're right with your questions:

    Two VMkernel Ports in the same subnet ?

    Sure, why not ? As long as you want to provide redundant storage paths that's the way to go.

    For a deeper understanding read the aforementioned document, it provides tons of useful info.

    Especially page 32ff. "Setting Up and Configuring Dependent Hardware iSCSI Adapters" and 37ff "Bind iSCSI Ports to iSCSI Adapters" are necessary to get the most out if it.



  • 3.  RE: Configuring iSCSI dependent hardware

    Posted Mar 19, 2011 12:02 PM

    OK... Thanks for your reply... So next week I try to make this changes. I've only this doubt yet.

    I have to create the two VMKernel ports linked with a nic and then I have to bind the vmk1 and vmk2 with the iSCSI adapter.

    My question is: Is there a correlation between the iSCSI adapter and the physical nic?

    thanks a lot for the answer



  • 4.  RE: Configuring iSCSI dependent hardware

    Posted Mar 20, 2011 09:19 AM

    having spent quite some time figuring this out, I believe the following might help.

    We use broadcom 5709 adapters with TOE and ISCSI offload, but the fine print did not mention that these are dependent adpaters which means they are driven by the VMware iscsi s/w adapter . and forget the option to set targets in the bios of the HBA - it does not work with ESXi.

    Despite this workaroundw its  still means that iscsi encapsulations occurs at hardware layer saving precious cpu cycles available to vms with the trade off that performance metrics are not available since the isci processing takes place beyond the vmkernel stack.

    This method will work best if the dependent HBA is visible in the list of storage adapters and requires diving into command line via local or SSH console.

    The overview of the process

    1. Enable the built in s/w iscsi adapter and make a note of the adapter name (eg vmhba32)

    2. Create a new vSwitch with the dependent HBA nics as uplinks

    3. Create one vm kernel port for each uplink ( this is important else the process will fail).

    4. Associate each vm kernel port with a single uplink moving the rest to unused (not standby).This is done via the vSwitch propeties(vSwitch properties-->Edit-->Nic Teaming tab and failover order)

    5. Open the console and check out if any physical links are currently associated with the s/w iscsi adapter (should be none)

         esxcli swiscsi vmnic list -d vmhba32

    6. Run thru the list of physical nics on the host
        esxcfg-vmknic -l
    7. Associate the target network inteface (dependant HBA uplinks) with the s/w iscsi adaptor
        esxcli swiscsi nic add -n vmk0 -d vmhba32

    8. Repeat for each HBA that you want to associate  (in our case it was 4)

    9. Double check association

    esxcli swiscsi vmnic list -d vmhba32

    10. And then open up s/w iscsi adaptor and enter the ip configuration of the target server.Rescan to detect luns and thats it.

    My reference was

    http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1025644

    However, the process does not work as decribed in the kb without enabling the sw adapter first. And setting targets in the HBA bios did not work.

    Hope the attached figure helps




  • 5.  RE: Configuring iSCSI dependent hardware

    Posted Mar 20, 2011 10:15 AM

    thanks for your answer but following this document: http://www.vmware.com/pdf/vsphere4/r41/vsp_41_iscsi_san_cfg.pdf at page 36 fig. 2-1.

    I have to bind the vmk port directly to my iSCSI hardware adapter and not to the iSCSI software adapter.... Am I wrong?

    thanks



  • 6.  RE: Configuring iSCSI dependent hardware

    Posted Mar 20, 2011 12:28 PM

    "I have to bind the vmk port directly to my iSCSI hardware adapter and not to the iSCSI software adapter.... Am I wrong?"

    - you are right in the interpretation of the guide .

    However,binding the underlying nic to the iscsi hardware adpater  did not work for me for example

    esxcli swiscsi nic add -n vmk1 -d vmhba34

    -->  always threw up an 'invalid adapter' error .  I think this works with iscsi independent HBAs not dependent HBAs. it certainly did not work with Broadcom 5709 adapters we have. 

    After a bit of fuddling about, i settled on the process is my last post to get it working.

    From what I could gather  (following on from KB 1025644) when i associated the underlying nic with the s/w adapter, the associated dependent iscsi hardware becomes linked with the storage stack.

    What model is your iscsi HBA ? and does it appear in the storage devices list of host ?



  • 7.  RE: Configuring iSCSI dependent hardware

    Posted Mar 20, 2011 01:53 PM

    Attached the iSCSI adapter see by vClient. I've installed HP NC382m that is drive by broadcom 5709S.

    My guess was to bind the first nic (vmk port) with vmhba32 and the second with vmhba33 and then select for each LUN the round robin path selection.



  • 8.  RE: Configuring iSCSI dependent hardware

    Posted Mar 20, 2011 03:26 PM

    "My guess was to bind the first nic (vmk port) with vmhba32 and the second with vmhba33 and then select for each LUN the round robin path selection."

    yep - that is correct at least in theor.

    Give it a go and would be interested on how it turns out.

    In our case, for reasons unknown, binding vmk port with the broadcom vmhba resulted in an error 'invalid adpater' . But  this was probably an exception .

    try your method first and do update with results



  • 9.  RE: Configuring iSCSI dependent hardware

    Posted Mar 20, 2011 06:52 PM

    I'm running with 4.1... You?



  • 10.  RE: Configuring iSCSI dependent hardware

    Posted Mar 20, 2011 08:44 PM

    Running 4.1 update 1



  • 11.  RE: Configuring iSCSI dependent hardware

    Posted Mar 21, 2011 08:33 AM

    I've seen the 'invalid adapter' error some times as well with extremely strange behaviors ...

    Sometimes a simple host reboot made the trick happen, other times i had to delete the vmk ports completely, reboot and recreate them from the CLI - GUI didn't work. Quite strange.

    Actually the KB shows 2 configurations with iSCSI:

    1. Let the BCM handle everything (no SW iSCSI)

    2. Let SW iSCSI handle everything except Offloading.

    Good Luck in sorting everything out :smileywink:



  • 12.  RE: Configuring iSCSI dependent hardware

    Posted Mar 20, 2011 08:44 PM

    actually the subnet bit depends on your storage.

    if you have EMC you need to use a different subnet for every SP port that is configured for iscsi.

    while with equallogics you can have multiple vmkernels on the same subnet.

    main recomendation would be to configure as per the vendors best practices. Otherwise you will have disconects or bad performance.



  • 13.  RE: Configuring iSCSI dependent hardware

    Posted Mar 19, 2011 12:47 AM

    have a look at this: http://www.yellow-bricks.com/2009/03/18/iscsi-multipathing-with-esxcliexploring-the-next-version-of-esx/ it might be useful for your understanding as well as multipath setup.



  • 14.  RE: Configuring iSCSI dependent hardware

    Posted Mar 21, 2011 09:14 PM

    As soon as possible I will update the thread.... Thanks to everybody!!



  • 15.  RE: Configuring iSCSI dependent hardware

    Posted Jan 17, 2012 05:17 PM

    with which the matrix works Broadcom - offloading? in your case, or with what you've done the test if it works?