Raw Device Mappings is greyed out

[Nnyan]

Hello everyone.  I am an EXSi/VMware noob so I'm a bit out of my comfort zone.  I have a Supermicro motherboard with an onboard SAS controller that is supported in EXSi 6.  I have eight 4TB drives attached to the SAS HBA and 2 SSD drives connected to the onboard SATA.  I was able to add all the drives as datastores and that works fine.  But I need to dedicate several complete drives to one VM and I have not been able to get this to work as the RDM option is greyed out no matter which VM I try.  I tried removing one of the Hard Drives from the datastores (it's gone) but I still don't see it as an option (and now I can't see it when I try to add it back as a datastore but that's another issue).

I've been researching this issue and I've tried disabling the RDMFILTER option in Advanced Settings and no go.  I tried flowing the process in KB article 1017530 (Raw Device Mapping for Local Storage I get the following error:

vmkfstools -z /vmfs/devices/disks/t10.ATA_____TS32GSSD25S2DM___________________________201009163653400631A4:9 /vmfs/volumes/APPSSD/RDM/rdm9.vmdk

Failed to create virtual disk: The system cannot find the file specified (25).

I thought maybe I needed to create the RDM folder in the store so I did it and then I got a new message:

Failed to create virtual disk: Operation not permitted (65545).

Any help would be greatly appreciated, thank you!

[Nnyan]

Not sure it makes any difference but they are 3TB drives on this server.

[vHaridas]

Does your SAS controller support below requirements?

To configure an RDM on local storage with ESXi/ESX hosts, both of these conditions are required:

• The candidate device must support SCSI Inquiry Vital Data Product (VPD) page code 0x83, which is used to directly address and communicate with RDM devices.
  For more information on SCSI sense code, see Understanding SCSI Check Conditions in VMkernel logs during rescan operations (1010244).
• The candidate device must export a global serial number, which enables the ESXi/ESX host to uniquely identify the device for use as an RDM

Also check this KB to check SCSI codes.

VMware KB: Understanding SCSI Check Conditions in VMkernel logs during rescan operations

Thanks,

Haridas

Virtual Admin

[Nnyan]

I'm not exactly sure how to find that out, I opened a ticket with Supermicro asking them that.  The motherboard is a Supermicro MDB-X8DT6-F -O with an additional Intel RS2WC080 HBA that has been flashed to IT (v20.  I haven't had a chance to flash the onboard LSI2008 to IT mode).

[Nick_Andreev]

Can you give us the output of the following command:

# ls -l /vmfs/devices/disks

[Nnyan]

Sorry for the delay everyone I had an unexpected trip out of state.  Here is the output:

[root@JEDI:~] ls -l /vmfs/devices/disks

total 44250959544

-rw-------    1 root     root     3000592982016 Jun  4 04:38 naa.5000c5004a189990

-rw-------    1 root     root     3000591450112 Jun  4 04:38 naa.5000c5004a189990:1

-rw-------    1 root     root     3000592982016 Jun  4 04:38 naa.50014ee0036f6ca1

-rw-------    1 root     root     3000591450112 Jun  4 04:38 naa.50014ee0036f6ca1:1

-rw-------    1 root     root     3000592982016 Jun  4 04:38 naa.50014ee0036f6e24

-rw-------    1 root     root     3000591450112 Jun  4 04:38 naa.50014ee0036f6e24:1

-rw-------    1 root     root     3000592982016 Jun  4 04:38 naa.50014ee058c4b1ca

-rw-------    1 root     root     3000591450112 Jun  4 04:38 naa.50014ee058c4b1ca:1

-rw-------    1 root     root     3000592982016 Jun  4 04:38 naa.50014ee058c4b211

-rw-------    1 root     root     3000591450112 Jun  4 04:38 naa.50014ee058c4b211:1

-rw-------    1 root     root     3000592982016 Jun  4 04:38 naa.50014ee0ae1a4795

-rw-------    1 root     root     3000591450112 Jun  4 04:38 naa.50014ee0ae1a4795:1

-rw-------    1 root     root     3000592982016 Jun  4 04:38 naa.50014ee0ae1a483c

-rw-------    1 root     root     3000591450112 Jun  4 04:38 naa.50014ee0ae1a483c:1

-rw-------    1 root     root     3000592982016 Jun  4 04:38 naa.50014ee0ae1a4953

-rw-------    1 root     root     120034123776 Jun  4 04:38 t10.ATA_____Corsair_Neutron_GTX_SSD00________________12357901000019090361

-rw-------    1 root     root     120030462464 Jun  4 04:38 t10.ATA_____Corsair_Neutron_GTX_SSD00________________12357901000019090361:1

-rw-------    1 root     root     32017047552 Jun  4 04:38 t10.ATA_____TS32GSSD25S2DM___________________________201009163653400631A4

-rw-------    1 root     root       4161536 Jun  4 04:38 t10.ATA_____TS32GSSD25S2DM___________________________201009163653400631A4:1

-rw-------    1 root     root     4293918720 Jun  4 04:38 t10.ATA_____TS32GSSD25S2DM___________________________201009163653400631A4:2

-rw-------    1 root     root     24095038976 Jun  4 04:38 t10.ATA_____TS32GSSD25S2DM___________________________201009163653400631A4:3

-rw-------    1 root     root     262127616 Jun  4 04:38 t10.ATA_____TS32GSSD25S2DM___________________________201009163653400631A4:5

-rw-------    1 root     root     262127616 Jun  4 04:38 t10.ATA_____TS32GSSD25S2DM___________________________201009163653400631A4:6

-rw-------    1 root     root     115326976 Jun  4 04:38 t10.ATA_____TS32GSSD25S2DM___________________________201009163653400631A4:7

-rw-------    1 root     root     299876352 Jun  4 04:38 t10.ATA_____TS32GSSD25S2DM___________________________201009163653400631A4:8

-rw-------    1 root     root     2684354560 Jun  4 04:38 t10.ATA_____TS32GSSD25S2DM___________________________201009163653400631A4:9

lrwxrwxrwx    1 root     root            73 Jun  4 04:38 vml.01000000003132333537393031303030303139303930333631436f72736169 -> t10.ATA_____Corsair_Neutron_GTX_SSD00________________12357901000019090361

lrwxrwxrwx    1 root     root            75 Jun  4 04:38 vml.01000000003132333537393031303030303139303930333631436f72736169:1 -> t10.ATA_____Corsair_Neutron_GTX_SSD00________________12357901000019090361:1

lrwxrwxrwx    1 root     root            73 Jun  4 04:38 vml.01000000003230313030393136333635333430303633314134545333324753 -> t10.ATA_____TS32GSSD25S2DM___________________________201009163653400631A4

lrwxrwxrwx    1 root     root            75 Jun  4 04:38 vml.01000000003230313030393136333635333430303633314134545333324753:1 -> t10.ATA_____TS32GSSD25S2DM___________________________201009163653400631A4:1

lrwxrwxrwx    1 root     root            75 Jun  4 04:38 vml.01000000003230313030393136333635333430303633314134545333324753:2 -> t10.ATA_____TS32GSSD25S2DM___________________________201009163653400631A4:2

lrwxrwxrwx    1 root     root            75 Jun  4 04:38 vml.01000000003230313030393136333635333430303633314134545333324753:3 -> t10.ATA_____TS32GSSD25S2DM___________________________201009163653400631A4:3

lrwxrwxrwx    1 root     root            75 Jun  4 04:38 vml.01000000003230313030393136333635333430303633314134545333324753:5 -> t10.ATA_____TS32GSSD25S2DM___________________________201009163653400631A4:5

lrwxrwxrwx    1 root     root            75 Jun  4 04:38 vml.01000000003230313030393136333635333430303633314134545333324753:6 -> t10.ATA_____TS32GSSD25S2DM___________________________201009163653400631A4:6

lrwxrwxrwx    1 root     root            75 Jun  4 04:38 vml.01000000003230313030393136333635333430303633314134545333324753:7 -> t10.ATA_____TS32GSSD25S2DM___________________________201009163653400631A4:7

lrwxrwxrwx    1 root     root            75 Jun  4 04:38 vml.01000000003230313030393136333635333430303633314134545333324753:8 -> t10.ATA_____TS32GSSD25S2DM___________________________201009163653400631A4:8

lrwxrwxrwx    1 root     root            75 Jun  4 04:38 vml.01000000003230313030393136333635333430303633314134545333324753:9 -> t10.ATA_____TS32GSSD25S2DM___________________________201009163653400631A4:9

lrwxrwxrwx    1 root     root            20 Jun  4 04:38 vml.02000000005000c5004a189990535433303030 -> naa.5000c5004a189990

lrwxrwxrwx    1 root     root            22 Jun  4 04:38 vml.02000000005000c5004a189990535433303030:1 -> naa.5000c5004a189990:1

lrwxrwxrwx    1 root     root            20 Jun  4 04:38 vml.020000000050014ee0036f6ca1574443205744 -> naa.50014ee0036f6ca1

lrwxrwxrwx    1 root     root            22 Jun  4 04:38 vml.020000000050014ee0036f6ca1574443205744:1 -> naa.50014ee0036f6ca1:1

lrwxrwxrwx    1 root     root            20 Jun  4 04:38 vml.020000000050014ee0036f6e24574443205744 -> naa.50014ee0036f6e24

lrwxrwxrwx    1 root     root            22 Jun  4 04:38 vml.020000000050014ee0036f6e24574443205744:1 -> naa.50014ee0036f6e24:1

lrwxrwxrwx    1 root     root            20 Jun  4 04:38 vml.020000000050014ee058c4b1ca574443205744 -> naa.50014ee058c4b1ca

lrwxrwxrwx    1 root     root            22 Jun  4 04:38 vml.020000000050014ee058c4b1ca574443205744:1 -> naa.50014ee058c4b1ca:1

lrwxrwxrwx    1 root     root            20 Jun  4 04:38 vml.020000000050014ee058c4b211574443205744 -> naa.50014ee058c4b211

lrwxrwxrwx    1 root     root            22 Jun  4 04:38 vml.020000000050014ee058c4b211574443205744:1 -> naa.50014ee058c4b211:1

lrwxrwxrwx    1 root     root            20 Jun  4 04:38 vml.020000000050014ee0ae1a4795574443205744 -> naa.50014ee0ae1a4795

lrwxrwxrwx    1 root     root            22 Jun  4 04:38 vml.020000000050014ee0ae1a4795574443205744:1 -> naa.50014ee0ae1a4795:1

lrwxrwxrwx    1 root     root            20 Jun  4 04:38 vml.020000000050014ee0ae1a483c574443205744 -> naa.50014ee0ae1a483c

lrwxrwxrwx    1 root     root            22 Jun  4 04:38 vml.020000000050014ee0ae1a483c574443205744:1 -> naa.50014ee0ae1a483c:1

lrwxrwxrwx    1 root     root            20 Jun  4 04:38 vml.020000000050014ee0ae1a4953574443205744 -> naa.50014ee0ae1a4953

[root@JEDI:~]

[christianZ]

Hello here,

local storage is basically not supported.

check this:

https://kb.vmware.com/selfservice/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=1017530

Reg

Chris

[Nnyan]

So do i need to install a different HBA or separate my storage onto a NAS/DAS type thing and then I can RDM?

[vijayrana968]

To configure a local device as an RDM disk:

1. Open an SSH session to the ESXi/ESX host.
2. Run this command to list the disks that are attached to the ESXi host:
   
   # ls -l /vmfs/devices/disks
   
   
3. From the list, identify the local device you want to configure as an RDM and copy the device name.
   
   Note: The device name is likely be prefixed with t10. and look similar to:
   
   t10.F405E46494C4540046F455B64787D285941707D203F45765
4. To configure the device as an RDM and output the RDM pointer file to your chosen destination, run this command:
   
   # vmkfstools -z /vmfs/devices/disks/diskname/vmfs/volumes/datastorename/vmfolder/vmname.vmdk
   For example:
   
   # vmkfstools -z /vmfs/devices/disks/t10.F405E46494C4540046F455B64787D285941707D203F45765 /vmfs/volumes/Datastore2/localrdm1/localrdm1.vmdk
   
   Note: The size of the newly created RDM pointer file appears to be the same size and the Raw Device it it mapped to, this is a dummy file and is not consuming any storage space.
   
   
5. When you have created the RDM pointer file, attach the RDM to a virtual machine using the vSphere Client:
   
   
   1. Right click the virtual machine you want to add an RDM disk to.
   2. Click Edit Settings.
   3. Click Add.
   4. Select Hard Disk.
   5. Select Use an existing virtual disk.
   6. Browse to the directory you saved the RDM pointer to in step 5 and select the RDM pointer file and click Next.
   7. Select the virtual SCSI controller you want to attach the disk to and click Next.
   8. Click Finish.
      
      
6. You should now see your new hard disk in the virtual machine inventory as Mapped Raw LUN.

Notes:

• As this virtual machine now has an attached local disk migration, using vMotion is not possible.
• If you need to remove the local RDM mapping from the virtual machine, just apply the same process you would with a shared storage RDM. In the vSphere Client: right-click the virtual machine, click Edit settings, select the RDM disk and then click delete from disk. This does not delete the data on the disk, it only deletes the RDM mapping file.

[Nnyan]

@vijayrana968

Thank you for the information!  but when I run that command I get the following error:

[root@JEDI:/vmfs/volumes] vmkfstools -z /vmfs/disks/t10.ATA_____TS32GSSD25S2DM___________________________201009163653400631A4:9 "/vmfs/volumes/APPSSD/DISK9.vmdk"

Raw device, '/vmfs/disks/t10.ATA_____TS32GSSD25S2DM___________________________201009163653400631A4:9' doesn't exist, or isn't a disk.

You can see the output of the ls -l command above.

Thank you for your time and help!

[vijayrana968]

Did you created pointer file correctly !

# vmkfstools -z /vmfs/devices/disks/t10.F405E46494C4540046F455B64787D285941707D203F45765 /vmfs/volumes/Datastore2/localrdm1/localrdm1.vmdk

[Nnyan]

No I did not!  I fixed the error and it created!  YES!  T H A N K   Y O U!!!

But...

Now I go to add it to a VM and I get this:

Reconfigure Virtual Machine

Incompatible device backing specified for device '0'.

Starting to google that but so far no traction.

(

[Nnyan]

ok not sure why but the VMDK file that was created is only 30gb in size where the disk is 3TB so I'm not sure if this is OK or an issue.

[Nnyan]

Ok not sure how I did this but one of my hard drives is not showing up as a data store.  I see it attached to the Storage Adapter (LSI2008) but under storage it's not listed and it doesn't show up when I "add storage" or do a rescan all.  I did notice this under the Storage Adapter listing for the disks:

-rw-------    1 root     root        2.7T Jun  9 03:43 naa.5000c5004a189990

-rw-------    1 root     root        2.7T Jun  9 03:43 naa.5000c5004a189990:1

-rw-------    1 root     root        2.7T Jun  9 03:43 naa.50014ee0036f6ca1

-rw-------    1 root     root        2.7T Jun  9 03:43 naa.50014ee0036f6ca1:1

-rw-------    1 root     root        2.7T Jun  9 03:43 naa.50014ee0036f6e24

-rw-------    1 root     root        2.7T Jun  9 03:43 naa.50014ee0036f6e24:1

-rw-------    1 root     root        2.7T Jun  9 03:43 naa.50014ee058c4b1ca

-rw-------    1 root     root        2.7T Jun  9 03:43 naa.50014ee058c4b1ca:1

-rw-------    1 root     root        2.7T Jun  9 03:43 naa.50014ee058c4b211

-rw-------    1 root     root        2.7T Jun  9 03:43 naa.50014ee058c4b211:1

-rw-------    1 root     root        2.7T Jun  9 03:43 naa.50014ee0ae1a4795

-rw-------    1 root     root        2.7T Jun  9 03:43 naa.50014ee0ae1a4795:1

-rw-------    1 root     root        2.7T Jun  9 03:43 naa.50014ee0ae1a483c

-rw-------    1 root     root        2.7T Jun  9 03:43 naa.50014ee0ae1a483c:1

-rw-------    1 root     root        2.7T Jun  9 03:43 naa.50014ee0ae1a4953

The item i highlighted in bold is the disk that doesn't show up under datastores (or available to be added).  It's the only one that doesn't have a second line that ends in a :1.  Not sure what to do to get this to show up again.

[Nick_Andreev]

Disk without a ":1" on the end means it's not partitioned. When you create a VMFS datastore on a LUN, vSphere creates the first partition. So basically that means you didn't create a datastore on this disk.

Can you see this disk when you run the Add Datastore wizard?

[Nnyan]

Ok that makes sense.  I used to have this hard drive as a datastore but I removed it (don't ask me how as I was attempting to direct attach it to a VM and after this it "disappeared").  When I run the process to add another datastore the disk isn't there (I can see it under the storage adapter but nowhere else) so I'm at a loss as to what to do next.

[Nick_Andreev]

Most likely ESXi servers now cannot determine what exactly is on this LUN and neither mount the file system, nor let you recreate it.

Did you try to delete the LUN on the array and simply create it again?

[Nnyan]

At this point I'm not sure what I did or how.  Which process do you recommend I follow?  I found this but I'm not sure if that's the appropriate steps in my situation.

Unmounting a LUN or detaching a datastore/storage device from multiple VMware ESXi 5.x/6.0 hosts (2004605) | VMware KB

[Nick_Andreev]

You have no datastores attached from this LUN. So all you need is to unmount the LUN. Then proceed with recreating the LUN on the array.

Alternatively, you can format the LUN from ESXi CLI.

[Nnyan]

I'm sorry for the long delay in response but I had a family illness and I was out of the country.  Anyway I'm back and I never did get this resolved and I see I never answered Nick_Andreev (sorry!).  I'll try look for how to unmount the LUN and or format it from ESXi CLI and see how that goes.