Should you segregate Network and storage traffic?

[MOC89]

To give some background we are using FCoE, and each server has 6 adaptors. The way it was planned was 2 for network, 2 for Storage and 2 for DMZ. The topic has been raised on using 4 for combined network and storage to increase throughput.

Could anyone point me to the best practices of this or advice if they feel it should be seperate or combined?

Thank you

[chriswahl]

I always advise using a disparate network for IP storage (or at least a unique VLAN), unless you have some really solid QoS that will guarantee the storage packets (as with something like a blade enclosure). The issue is that storage is much more sensitive to latency than (most) network traffic, and contension could cause significant issues.

http://blogs.vmware.com/networking/2011/11/vds-best-practices-.html

IP storage traffic is carried over vmknic ports and this traffic varies according to disk I/O requests. With end-to-end jumbo frame configuration, more data is transferred with each Ethernet frame reducing the number of frames on the network. This larger frame reduces the overhead on servers and targets and improves the IP storage performance. Congested and lower speed networks can cause latency issues that disrupt access to IP storage. It is recommended to provide high-speed path for IP storage and avoid any congestion in the network infrastructure.

[weinstein5]

Best practice is to seperate the different type of network traffic - management, vmotion, vm, and storage - whether iSCSI, NAS or FCoE

[rickardnobel]

This six NICs, are they 10 Gbit?

[logiboy123]

Any design decision is a rationalization of trade offs. The more NICs you can assign to a single vSS or vDS the more redundancy you gain, but the greater the risk of security of performance issues.

If you have Ent+ you can assign several NICs to the same vDS and use Load Based Teaming along side Network IO Control with traffic segmented using VLANs.

Please check out my blog for some design diagrams and information.

http://vrif.blogspot.com

Regards,

Paul

[Texiwill]

Hello,

Security best practices are to segregate your storage network from all else, separated would be best.... What is the difference? Segregated is a logical separation while separated to me is a physical separation (i.e. different pNICs for storage traffic). However, with 10G adapters being common place it is often wise to use segregation but allow the storage a definitive QoS down to the hardware. Unfortunately this would be for vmkernel devices and the only way I know how do to this is with NetIOC egress QoS (requiring Ent+ and a dVS).

Best regards,
Edward L. Haletky
VMware Communities User Moderator, VMware vExpert 2009, 2010, 2011, 2012

Author of the books 'VMWare ESX and ESXi in the Enterprise: Planning Deployment Virtualization Servers', Copyright 2011 Pearson Education. 'VMware vSphere and Virtual Infrastructure Security: Securing the Virtual Environment', Copyright 2009 Pearson Education.

vSphere Upgrade Saga -- Virtualization Security Round Table Podcast -- The Virtualization Practice

[rickardnobel]

Edward Haletky wrote:

but allow the storage a definitive QoS down to the hardware. Unfortunately this would be for vmkernel devices and the only way I know how do to this is with NetIOC egress QoS (requiring Ent+ and a dVS).

Even if not having the Distributed vSwitch version 5 with QoS support it is typically available in the switch hardware. Even if the dVS does set the 802.1p priority bits to some value it does not have to be respected by the physical switch and even if not set you could often create priority rules for the incoming ports.

This could be set on both certain VLAN id:s, but also on things like TCP ports and similar.

[MOC89]

Cheers guys, much appreciated!