Best practices for iSCSI configuration

[cool_guy2]

I inherit the following vmWare setup: EMC iSCSI storage with 4 nic cards (as of now only two nics are in use). Two ESXi host with vSphere 5 standard. Each host has 2 nic cards with 4 port each (one Broadcom card, one Intell) with only 2 ports are in use from each server.

I am trying to redesign the configuration and I am wonderring what is the best way of doing it. Also I am not sure if I need to have two identical nic cards for iSCSI configuration?

Thank you in advance for all your help.

Serge

[memaad]

Hi,

Page number 127 http://pubs.vmware.com/vsphere-51/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-51-storage-guide.pdf

Regards

Mohammed

[cool_guy2]

After reading all the articles and some ofthe white papers, the following is the configuration that I am thinking (Assume that my Broadcom card has nic ports assigned as eth0 through eth3 and Intel card has ports assigned as eth4 through eth7):

vSwitch0 --> eth0, eth4 (for managment and vMotion in Active/Stanby configuration). Let's say I will use VLAN 100 for management and VLAN 101 for vMotion

vSwitch1 --> eth1, eth2, eth5, eth6 (for iSCSI connection to the storage in Active/Active configuration) -- VLAN 102

vSwitch2 --> eth4, eth7 (for running VMs in Active/Active configuration) -- VLAN 103

Will that configuration works?

There are also some questions that I can;t find answers for: since i am using Broadcom and Intell NICs, would that cause problem with iSCSI? Also how would I need to handle issue with multipathing?

Serge

[LennieDH]

As first thing and generally speaking I would plug and configure all ports on each server and all ports on storage, this means not to connect random cables between ESXi hosts, storage and rest of the network but making a little project accordingly to your current environment.

I suppose that to achieve a good configuration we should know more details of your environment, for example how many VMs are running, if you have set up a cluster for your VMs, what kind of EMC storage is used (VNX, Symmetrix,Celerra, ...) and what kind of load it has to withstand...etc.

A good startup point is to set-up hardware correctly e.g. making sure all nics are used (more nics could mean more throughput if everything is fine configured) ensuring there are no single point of failure within your environment (i.e. redundant network links between your ESXi hosts - storage and between storage - rest of the network, ensuring everything is fine tuned within your EMC storage,etc.).

Next step is to achieve performance by applying best practices...a good start could be configuring network in your ESXi hosts, using vDS, implementing VLANS (or verify if there are some already there), enabling mechanisms that allow more throughput like,for example, jumbo frame, TSO, etc...

This is just my opinion...maybe someone more expert than me can advice you better!

Hope this could help you,

Regards.

[cool_guy2]

In terms of storage its EMC VNXe 3100, it doesn't have to withtand a lot of load.

ESXi host will be set up in a cluster with virtualize vCenter with SQL Express database. I can't set up vDS as Standard edition do not support it.

[LennieDH]

Ok, so with Standard Edition you restrict the things to look at (i.e. no FT, no DRS).

A good start point for iSCSI is the guide meemad linked above.

Other software related aspects I would consider is enabling jumbo frames between VNX and ESXi hosts (and switches within if any) configuring VLANs (separate VLAN for iSCSI traffic, management traffic, rest of traffic), using separate vmnics, if possibile, for iSCSI, management, vmotion and VMs network.

Edit: VLAN separation is fine.

nic distribution seems ok, maybe have a look at nic teaming policy what best suits you especially in iSCSI vlan and if possible to separate vm kernel ports for vmotion and management.

Hope this could help,

Regards.

[cool_guy2]

Thank you for the reply. I can't find any suggestions for nic teaming for iSCSI. Also how do I enable jumbo frames in vSphere? Another questions that noone is answered: would two different NIC brands cause any issues with iSCSI?

Serge

[memaad]

Hi,

Here is KB article for Jumbo Frame enable http://kb.vmware.com/kb/1007654

Also VMware dont recommend having two different brand nic in same teaming.

Also need to have one nic binded to vmkernel port while using for iscis and other nic to be in unused state.

Also your configuration look good.

Regards

Mohammed

[Rubeck]

Also VMware dont recommend having two different brand nic in same teaming.

For real?  Please provide a link showing this....

/Rubeck

[memaad]

Hi ,

Sorry ,I am not able to find any document, thats was I recall from past.

Regards

Mohammed

[Rubeck]

Hi Mohammed..

No need to be sorry at all.....

Though you were just about to crack one of those reasons of mine for being in love with the VMware hypervisor for many years..;-)

Im just a simple end-user so please bare with me...

To OP: Sorry for buggin' in... 

/Rubeck

[Gkeerthy]

general design if you have many nics..you can follow the below

vSwitch0  --> eth0, eth4 (for managment and vMotion in Active/Stanby  configuration). Let's say I will use VLAN 100 for management and VLAN  101 for vMotion, use multi nic vmotion

http://pibytes.wordpress.com/2013/01/12/multi-nic-vmotion-in-vsphere-5-x/

vSwitch1 --> eth1, eth2, eth5, eth6 (for iSCSI connection to the storage in Active/unused configuration) -- VLAN 102 create 4 vmkernel portgroups, each pnic will be active and other 3 will be unsed, there is no need to use the esx vswitch failover.. the psp will handle the failover,,, that is the it is depend on the multipathing policy and it will take care of the uplink failures.

vSwitch2 --> eth4, eth7 (for running VMs in Active/Active configuration) -- VLAN 103

there is no issue in using pnics of different vendors.. we also normally use this in the design. This will prevent the firmware/driver errors which may raise in the due course. So if you use intel & broadcom in the same vswitch it will be very good.. and will be a roboust design.

Jumbo frame you can use the value 9000, and it can set in the vswitch and pswitch.

[cool_guy2]

in this configuration to I need to do iSCSI binding for 4 vmkernel portgroups?

[Gkeerthy]

yes.. you need to bind the 4 iscsi vmkernel poert groups...