Updated Pros and Cons for NFS vs iSCSI for datastores.

[WaveJam]

Hi..

I've seen a few comparisons of NFS vs iSCSI but they are older threads. Does anyone have an updated pros and cons list for NFS vs iSCSI which applies to vSphere 4.x / 5.x ?

Thanks.

[J1mbo]

The real difference is that with iSCSI, the file system is at the vSphere host end (VMFS), whilst with NFS the file system is at the other end (i.e. the storage server).  On the wire, packet overheads for block access is almost identical.

NFS has the real advantage that the NFS server process controls what's going on; it's very easy to hook up new hosts (just an nfs share mapping obviously) and similiarly pretty difficult to damage the file system (since the NFS server decides what is an isn't allowed).  Also there are no capacity limits, i.e. a 12TB NFS datastore can be mounted even on v4 (although there remains the irritating 2TB VMDK size limit).  When a file is deleted, just like on a Windows file server, the space is immediately available for something else.  VMDKs on NFS are also naturally thin provisioned (although they can be manually inflated, if required).

iSCSI on the other hand is a co-operative approach (a bit like co-operative multitasking in Windows 3) - there's no real gatekeeper over what is going on and access control is simply "you can, you can't".  So for example a Windows machine that has been granted rw permissions to a LUN could potentially damage it and the iSCSI target would be quite happy; this is the point - the iSCSI target (server) is dumb in the sense that it has very little knowledge of what it's sharing; it's just block access.  Write to block zero?  Permission to do so?  Yes - that's fine.  By-bye to whatever file system is on it.

Another problem with iSCSI is that where the storage vendor enables thin-provisioning at the storage layer, those LUNs tend to grow over time and it's very difficult to reclaim that space on the underlying storage since there is essentially no delete command - after all SCSI is an ancient protocol for 8-bit spinning disks that's been tunned over IP, so why would there be.  After a period of years this can lead to quite high proportions of storage being masked out and effectively unusable.

OTOH certain applications are explicitely not supported from NFS storage (Exchange 2010 database servers for example), although in practice we all know it will work perfectly well.

Hope that helps.

[BillMoller]

If NFS is not block level, like iSCSI, isn't it copying whole files back and forth over the wire to perform reads/writes?  i.e. under this assumption, NFS for large databases would be prohibitively slow?  Wouldn't it? 

[sparrowangelste]

NFS doesnt copy entire files back and forth.

it does block writes but instead of the write being controlled by the client, in NFS the client requests that storage to update the blocks

 example, a client WRITE
   request may cause the server to update data blocks, filesystem
   information blocks (such as indirect blocks), and file attribute
   information (size and modify times).  

http://www.ietf.org/rfc/rfc1094.txt

[BillMoller]

So iSCSI and NFS are basically the same but the filesystem is on the host device for NFS, whereas for iSCSI, the filesystem resides on the client?

[aravinds3107]

Welcome to the Community..

Take a look at the NFS Deepdive section here . Its a 4 part series worth checking it