Best way to connect a SAS Tape drive to a Guest?

[jbWiseMo]

I am planning to connect a SAS tape drive to a VM in our ESXi 4 server (on certified hardware). I expected this to be trivial, but as I researched the proper way to do it, I found a lot of confusing info, including some who claim it doesn't work at all.

So what is the proper way to connect a SAS Tape drive (LTO 4) to a Guest on an ESXi host, given these facts:

• VMDirectPath is not supported on the AMD processor model in use (because ESXi4 insists on IOMMU use)

• Performance of the tape transfer is important and needs to be stable for production use

• This is a tape drive, NOT a tape library

• I am not asking about backup software, that is a separate issue

And these rumors (unconfirmed):

• SCSI pass through to tape drives has problem

• Finding Tape compatible SAS controllers (not Disk SAS controllers) on the HCL is amazingly difficult

• Some people suggest getting an iSCSI to SAS converter and putting it on one of the 1Gb Ethernet ports, but I have seen no good suggestions for reasonably priced iSCSI to SAS converters

[kjb007]

The SCSI pass through is limited to certain hardware, or was limited to a very small set of hardware wtih 3.5 If you already have the LTO drive, then I would suggest to add it as a generic scsi device to your guest. Your mileage will vary, but I've seen it work with tape drives more often than not.

-KjB

[jbWiseMo]

This is the ESXi 4 forum, not the ESXi 3.5 forum, there may be differences.

I am aware that using SCSI passthrough as a "generic device" is probably the user interface option to use, IF it can be done with a SAS adapter on the ESXi 4u1 host.

The problem is which solution should I choose and if so which adapter should I buy:

• A PCIe SAS adapter for the ESXi host? (Using SCSI passthrough as a "generic device")

• An iSCSI to SAS adapter sitting between the host and the drive? (Using an iSCSI initiator in the Guest OS)

• A 3rd kind of solution which I haven't thought of?

[VMmatty]

I've done SCSI passthrough on regular SCSI tape changers in the past but the results have never been all that great. It does work but is especially limiting if you have multiple hosts in a cluster and you want to do things like VMotion.

I've never used a SAS to iSCSI bridge but I've used quite a few SCSI to iSCSI bridges and have found the solutions to work well. You're limited by the bandwidth of the gigabit link but in general I've seen similar performance numbers when compared to directly attached tape drives on physical servers. It also offers lots of flexibility in terms of VMotion, having multiple backup servers, etc. I really like the solution but you should understand that it raises the complexity of the environment. We've had situations where troubleshooting backup issues takes significantly longer than normal as vendors point fingers at each other.

This is the only SAS to iSCSI converter that I have heard of but have never used it myself.

http://www.4bridgeworks.com/products/potomac_iscsi.phtml

[jbWiseMo]

Update:

I purchased a VMWare certified SAS HBA from the same OEM (HP) as the server and tape drive. But so far, it doesn't work and I have posted a new thread about the specific way in which access to the tape drive from a VM fails. This was the only external SAS HBA which was VMWare certified and which the vendor recommended for that tape drive (HP recommends a choice of 3 HBAs for SAS LTO-4 drives, one is not on the VMWare HCL, one is for internal drives only, and the last one I bought).

I am now trying to look for iSCSI to SAS bridge prices, or a cheap but fast box to act as a unix/Linux rmt tape server (also over IP, but not iSCSI).

But I am still open to other ideas.

[AndreTheGiant]

Consider to use VMDirectPath or convert the SAS to iSCSI (several autoloader have this option).

Check also if your backup software is supported in virtual (it will work, but this does not mean that the vendor will support it).

Andre

[jbWiseMo]

As I said in my original post, VMDirectPath is not available due to ESXi v4u1 overreliance on the rare AMD IOMMU, which is not on this VMWare certified server.

And this is not an autoloader, it is a lone tape drive large enough to use only 1 tape/day (at which point it is carried offsite anyway).

Anyway, that other thread about SCSI pass-through failing was already answered by myself: It turns out that ESXi v4u1 SCSI pass through mishandles large block sizes, reducing the tape block size in the backup software solved the issue. This is undocumented because VMWare "support" insists that tapes are not supported at all...

[JCPanosh]

Using ESXi 4.1 on Dell servers we have installed a 6Gb SAS-HBA card into the server and used Hardware Pass Through.  The Tape Library is Dell TD-2000 and has LTO5 drive in it, so this is a library not a standalone drive, but the same circumstances would be the case for standalone drive as drive is connected to the SAS-HBA.

This system has been in place, without issues for >12 months. Performance hass been very good with transfer rates >2.5Gb/min.  The throughput hass been limited due to our backup SAN only having 2 * 1Gb NICs.

Last month wee changed the SAS-HBA for a SAS-iSCSI bridge.  This has also worked without incident but the throughput has dropped significantly because of the 2 * 1Gb NICs in the bridge rather than the 6Gb SAS interface in the SAS-HBA card.  I would recommend that 10Gb NICs are the minimum requirement for SAS-iSCSI tape integration.

[jbWiseMo]

This is just an update to let you all know that the configuration I installed is still working fine, now with the VMware kernel upgraded to ESXi 4.1 (This was done before ESXi 5 came out, hardware compatibility with ESXi 5.x not yet checked).

Basically, the host contains an additional SAS SCSI HBA used only for the tape drive (performance and stability choice), the tape drive is passed through as a genereic SCSI device to a dedicated virtual LSI SAS adapter in the VM that runs the backup software.  The SCSI address of the tape drive on the virtual adapter is manually set to the same one as what vClient reports it to be on the physical adapter, so any reference to the unit/lun in the SCSI commands won't need translation.  Additionally, the tape block size has been artificially set to a lower value to avoid the issue seen in ESXi 4.0 (no need to retest larger block sizes with later ESXi versions, turns out the disk SAN is the speed bottleneck during backups, not the tape interface).

Two small problems have been observed and need routine working around, but it works well enough for production:

• When powering up the ESXi 4.x host and/or the tape using Guest VM, a tape must be already in the drive or hardware detection fails.
• If an attempt is made to use the tape drive while no tape is inserted, the Guest VM hangs on the SCSI request and the physical SCSI bus is put in an invalid state which cannot be cleared without physically power cycling the host, which is disruptive but can be avoided with diligent operation procedures to make sure the tape drive is not empty at the scheduled times.

These two issues are not for discussion here, I just mentioned them for completeness.

Hardware used:

• HP Proliant Server with non-IOMMU AMD CPUs, model is on the ESXi 4.x HCL.
• HP SCSI adapter, which is both on the ESXi 4.x HCL (as a general SCSI adapter) and the HP HCL (for the tape drive and server).
• HP LTO-4 Ultrium tape drive with external SAS interface cable.  Tape is HH (half height) in an external enclosure.
• An external iSCSI disk array (not HP), model is on the ESXi 4.x HCL.

[Ryan_Veino]

Has anyone looked at or used this: http://www.avax.com/iSCSI_bridge.html

[jbWiseMo]

Yes, see above.  When we looked at them, they were simply too expensive for the job.  But JCPanosh did use such a bridge successfully.

Currently, we have upgraded our system from VMWare SCSI pass through (which tended to hang as previously explained) to a low cost headless and mostly diskless server which runs a minimal Linux installation and a script to buffer data before passing it to the SCSI tape in bursts of a few GB.  Basically, we did a Virtual to Physical conversion to work around a VMWare bug.