8 port switch recommendations for iSCSI network?

[VoxMedica]

I’m currently drafting up a plan for replacing our current infrastructure:

• VMware ESXi 5.0 (17 VMs, mix      of W2k8R2 file, Exchange, SQL, IIS)
• 3 x G1 HP BL460c G1 blades      in a HP C3000 Chassis
  • 96GB       Total RAM (32GB per Blade)
  • 55GHz       total CPU (2 x Quad Core Xeon E5345 per blade)
• EMC NX4 iSCSI SAN

With the following:

• Vmware ESXi 5.1
• 2 x HP DL380p Gen8 Mount      Servers
  • 182       GB Total Ram (96 Per host)
  • 2       x Octocore CPU’s per host
• New iSCSi SAN (probably a EMC      VNXe)

We currently have iSCSI traffic on the HP blade chassis switch on a seperate VLAN. For the upgraded infrastructure I am leaning towards using cross over cables for both:

• iSCSI between the hosts and SAN
• vMotion between the hosts

The main reason for doing so is that I can’t seem to find a decent cost effective 8 port switch that supports Jumbo Frames and Flow Control. From my research both are highly recommended for iSCSI networks, and the majority of switches in the 8 Port range don’t support both. The closest I found was the Dell PowerConnect 2808 Switch. Would this be ok for an iSCSI network? It’s drastically cheaper than the 24 port HP and Cisco models I was originally looking at, the price of which lead me down the path putting the money spent on switches towards more NICs in each host and using cross over connections instead.

Should I stick with my idea of using cross over connections? Or would the Dell or another relatively cheap 8 port switch be ok?

[vmnomad]

If your are not going to expand up your infrastructure in the near future I would say cross over cables are fine to use. The only reason why you would need a switch in your situation is to have some scalability.

I assume you will get small storage for only 17 VMs and so I think Jumbo Frames will be of very little use. Very obvious that number of disk spindles, RAID type proper iSCSI traffic load-balancing will have more influence on overall storage performance. That's why Jumbo Frames can be excluded from the equation.

Ae you buying 10GB NICs?

[VoxMedica]

I definitely don’t see us expanding past the two hosts unless my company goes through explosives growth. I configured each host so that all our VMs can safely run on one host if needed

The VNXe is currently spec’ed with:

• 2  x trays of 300GB 15K SAS drivers
• 1 x  Tray of 1TB SATA drivers
• 4 1GB NICs

As for the host networking, we were planning on each hosts having a 4 1GB NICs (1 quad port) since the Cisco switches we had planned to use for the iSCSI network were close to $7K. Now that I’ve removed them from the picture, I’m thinking of doing either:

1. 2 x  Quad Port 1GB NICs per host
   • 2 x vMotion
   • 2 x iSCSI
   • 4 x Network
2. 3 x Dual Port 10GB NICs per host
   • 2 x vMotion
   • 2 x iSCSI
   • 2 x Network

I don’t have concrete pricing on what it would cost to add the 10GB Nics to the hosts, but a quick search on the HP website reveal around $2K-3K extra per host. Currently I have no 10GB in my infrastructure, so the only place it would run at full speed would be the vMotion cross over network between the Hosts. I haven’t checked to see if the VNXe even offers 10GB NICs.  But it would be nice for future growth.

[vmnomad]

Both options are fine, but with option 1 you might want to change number of iSCSI ports as storage traffic  is more sensitive to available bandwidth. It is probably a good idea to check current storage and networking performance stats.

[VoxMedica]

Good point! After thinking about it some more i'm wondering if going fiber channel between the host and SAN would make more sense. Currently our VNX is configured with 8 1GB ports, but i'm checking for the cost different between 8GB Fiber channel vs 10GB Ethernet on both the host and SAN.

[Moto110]

@VoxMedica - I'm looking at doing the exact same configuration 2 x HP Gen 8 servers and the VNXe.  What did you decided to do about the iSCSI switch?  Did you buy one or did you just connect the servers directly to the SAN?  Can you explain how you finally decided to allocate your NICs?  Number of iSCSI vs Network vs vMotion ports?

Thanks.