Question about VMKernel iSCSI traffic and VLANs

[GeneNZ]

Hi there,

This is very basic question that I'm pretty sure I know the answer too, but I want to ask it anyway just to reassure myself. As a precursor to my question, the configuration of my ESX infrastructure is best outlined here: . Or more precisely, we have dual controller MD3000i's. Each controller has two ports, and each port is configured on two different subnets, with each subnet connected to different switch. ESX hosts are connected to both switches. The only difference to the guide is we have two MD3000i's configured the same, connecting to the same switches. iSCSI ports on each MD is configured on the same subnets, but different IP addresses.

At present, we are in the process of upgrading both our iSCSI switches from lowly Dlink DGS-1224T's to Cisco 2960T's. The switches were and will continue to be dedicated for iSCSI traffic, however, I am in the process of putting in place VLAN's on the switch-side. Originally we used the default VLAN on the switches, however, having added another MD3000i, it has been mentioned by Dell Support that best practise is to separate the iSCSI traffic of each MD3000i onto its own subnet and VLAN. This would result in 4 iSCSI VLANs, two on each switch and two for each MD3000i. Firstly, is this indeed best practise?

Secondly, if I proceed with the above migration to 4 iSCSI VLANs, as each switch port is going to be effectively an access port, will there be any need to fill out the VLAN ID field in the VMKernel configuration page? Presumably, this field is used when VLAN tagging is used, but since our switches have no need to trunk to any other switches (as they are dedicated for iSCSI traffic), should there be any need to fill it in? I would assume that it would be safe to keep the two existing subnets, create two new subnets and make changes to one MD3000i and connecting ESX hosts. Provided the switch and switch ports has been appropriate configured with the right VLANs, the rest should be transparent and there would be no need to provide VLAN info to the ESX hosts at all?

Would be great to get some answers, and I thank you in advance!

Gene

[naveenvm]

1) Yes it is as per the best practices for ESX iscsi, having separate network and vlan for the iscsi traffic.

2) Absolutely no, you do not need to mention anything in the vlan field if you are using access port. Its rather a mandatory thing than a choice. If you provide the vland id with access port, it will loose connectivity.

Please clarify a bit, why you need to create two different vlans for each MD3000i. Are you going to use multiple iscsi storages on the same ESX box? or, you are going to use only one iscsi and use these 4 ports for the same single VMkernel interface?

NUTZ

VCP 3.5

(Preparing for VCP 4)

[GeneNZ]

Thanks for your answer.

The reason I would need two VLANs for each MD3000i, is because each MD3000i has two subnets associated with it. My plan for the VLAN design is as follows

MD3000i 1:

iSCSI subnet 1: 192.168.100.0/24 - VLAN 100

iSCSI subnet 2: 192.168.101.0/24 - VLAN 101

MD3000i 2:

iSCSI subnet 1: 192.168.200.0/24 - VLAN 200

iSCSI subnet 2: 192.168.201.0/24 - VLAN 201

What this means that each MD3000i controller has two iSCSI host ports. One host port on each controller will have an IP address configured in one subnet, and the other host port in another. The result is 4 iSCSI VLAN's altogether (since there are four subnets), or 2 iSCSI VLANs on each switch. My plan is actually to use the native VLAN 1 for switch management onto our main network.

I probably have worded it incorrectly which has resulted in some confusion. I should have really said there would be 2 iSCSI VLAN's per switch, not including the default VLAN which I'm assigning for switch management.

Thanks again,

Gene

[rolohm]

You seem to be on the right track when it comes to the iscsi setup. To me it looks as if you will need to dedicate four physical interfaces per esx to iscsi traffic if they are supposed to reach all vlans but if that's the intention then there is no problem with that. What actually beckons me to contribute to the discussion is that you plan to use VLAN1 as management. That's traditionally a bad idea due to security reasons. See below link and reconsider that decision :smileyblush:

/R

[GeneNZ]

Thanks for the reply,

Actually the ESX hosts themselves will only access one of two MD3000i's. There is no need (and I don't think its supported by the MD3000i anyway) to have one ESX host connecting to both MD3000i's. Therefore we only dedicate two physical interfaces for iSCSI traffic. The creation of the VLAN's is really just to separate the traffic from each other.

Thanks for the link with regard to using VLAN1 for management. I will update my plan, and change the VLAN used for management traffic.

Thanks again.

[SomeJoe7777]

It is indeed best practice to use VLANs to separate iSCSI traffic from other kinds of network traffic (e.g. main network, management network, VMotion network, etc.). But I see only very few reasons for separating two iSCSI networks using VLANs.

Perhaps if ESX server "A" needs to talk to iSCSI storage "A", and ESX server "B" needs to talk to iSCSI storage "B", and "A" and "B" equipment is owned by different companies, then maybe you want to use VLANs to separate them for security purposes. In a similar situation, you might want to completely separate the two if you want to maintain two separate ESX infrastructures that are purposed differently.

But if you own and manage everything together, I see little reason to separate the two. VLANs separate all traffic from each other, but I would believe that to be counterproductive here. Although right now you want each ESX server to only connect to one MD3000i unit, there may come a time where that ESX server needs to connect to both units (to increase storage, or to enable VMotion for all VMs across all ESX hosts). By using only 2 subnets (one on each switch), a single ESX host can indeed connect to both MD3000i units, and access all the LUNs in both places. In addition to allowing VMotion for any VM to any host in the cluster (regardless of which MD3000i it was on), I believe this also allows Storage VMotion to move a running VM from one MD3000i to the other.

Also, since each ESX host will only be connected to the iSCSI network with 2 physical uplinks, separating the traffic via VLANs won't improve performance (you would need 4 uplinks for that). And the Cisco switches are not affected one way or the other -- they will switch the traffic between the proper devices/ports whether the ports and traffic are VLANed or not.

[GeneNZ]

Thanks for your reply, you definitely have given me food for thought. Funnily enough, the reasoning you outlined quite succinctly was the original reason why I placed the two MD3000i's on the same subnets in the first place. The idea was that if I wanted to chop and change the ESX hosts between MD's, it could be done with minimal hassle, since all I would have to do is change the iSCSI targets - no fiddling around with changing VMKernel IP addresses etc.

It was only when I contacted Dell Support due to a problem, that they were insistent that best practise is to separate the iSCSI traffic on different VLANs. I couldn't fault them on this argument, and they used that argument to try get out of their obligations. Now that we're on the verge of purchasing new switches, I wanted to separate the traffic so any future dealings with Dell Support can't result in them using this argument against me. Plus I figured there was another unknown reason to me that the best practise was chosen the way it was. I understand why you would want to segregate normal traffic from iSCSI traffic in separate VLANs, but was always unsure why you would need to segregate iSCSI traffic from other iSCSI traffic.

Still your reasoning has given me something to think about, since I figured I might be the only one who thought the way I did, and that best practises are there for a reason. But now I have reasoning to think otherwise. Thanks again, your post has been most useful.

[rolohm]

One reason for using VLANs is that they also, obviously, block broadcast storms. If you have two IP subnets for iscsi redundancy and you don't separate them by VLAN. If one interface on the "big ethernet lan" goes mad and start broadcasting crazy, then the built in multipathing for redundancy is of no use since the layer2 is jammed anyway.

/R

[GeneNZ]

In our case, broadcast storms are unlikely, primarily due to the fact that we have no trunking ports connected to the iSCSI switches and those switches are isolated. Also note that each switch it responsible for one iSCSI subnet only (not both). My understanding is broadcast storms are typically created due to redundant trunk links between switches and improperly implemented STP configuration, or through DoS attack. I'm sure there are other situations when broadcast storms can occur, but I understand these to be the most common. In our case, our iSCSI switches are independent from each other and our primary network, and run entirely in isolation. That is, of the two iSCSI switches, both are completely unaware of the other's existence or any other network, other than the single iSCSI subnet assigned to it. Also all ports are configured as access ports.

Hypothetically, if a broadcast storm were to occur somehow in our configuration, the redundancy should still work without two iSCSI VLANs per switch (how I see it). Assume one switch goes down due to broadcast storm - in effect the same as turning off the switch. Because the other redundant switch is isolated, both ESX cluster (one per MD3000i) should failover to the redundant switch. One valid point you do bring up is that if I were to separate both independent ESX clusters into their own VLAN, if a broadcast storm were to occur, it would mean only one cluster would have to failover to its redundant link, while the other cluster would continue to operate optimally. Still, this requires a broadcast storm to occur in the first place... which I struggle to see happening anyway, since they are isolated switches.

Regardless, thanks for your reply, it helps me consider ideas I haven't yet thought of.

Gene