Find the physical GPU a VM is connected to

[Gabrie1]

Hi,
Is there a way to find out to which physical GPU in the host a VMs is connected to?

Using $esxi.ExtensionData.config.GraphicsConfig.DeviceType against a host, will list the device IDs of the physical GPU.

DeviceId     GraphicsType
--------     ------------
0000:25:00.0 sharedDirect
0000:59:00.0 sharedDirect
0000:99:00.0 sharedDirect
0000:c5:00.0 sharedDirect

On the VM I configure a PCI device to use the "NVIDIA GRID vGPU" profile, but I can't figure out how I can check which of the physical GPUs in the host (we have 4 NVIDIA T4 adapters per host), the VM is assigned.

Any tips?

[Gabrie1]

Already figured it out, I was looking from VM to host, but when going to the host and zooming in on the GPU cards, you can also get the VMs connected to that GPU. I needed this because we have VDI VMs on ESXi7 that have NUMA pinning and we want to check if the GPU it is using is remote.

Function Get-VMbyID {
    param(
        $vmid
        )
    
    ## Find VM by ID
    $VMName = Get-VM -id "VirtualMachine-$vmid"
    $NUMANodeAffinity = $VMName | Get-AdvancedSetting -name "numa.nodeAffinity"
    Return $($VMName.Name, $NUMANodeAffinity.Value)
}

$cluster = get-cluster | Sort-Object name | Out-GridView -Title "Select cluster" -OutputMode Single
$ESXiHosts = $cluster | Get-VMHost | Sort-Object Name
$Report = @()
Foreach( $ESXi in $ESXiHosts)
{
    $ESXiGPUs = $ESXi.ExtensionData.config.Graphicsinfo
    $GPUCounter = 0
    Foreach( $ESXiGPU in $ESXiGPUs)
    {
        $GPUVMs = $ESXiGPU.vm
        Foreach( $GPUVM in $GPUVMs)
        {
            $row = "" | Select-Object ESXiName, devicename, vendorname, PciID, GraphicsType, MemorySizeInKB, vmname, vmid, numanode, GPU
            $row.ESXiName = $ESXi.Name
            $row.DeviceName = $ESXiGPU.DeviceName
            $row.VendorName = $ESXiGPU.VendorName
            $row.PciID = $ESXiGPU.PciID
            $row.GraphicsType = $ESXiGPU.GraphicsType
            $row.MemorySizeInKB = $ESXiGPU.MemorySizeInKB
            $row.vmid = $GPUVM.Value
            $row.vmname, $row.numanode = Get-VMbyID -vmid $GPUVM.Value
            $row.GPU = $GPUCounter
            $Report += $Row
        }
        $GPUCounter++
    }
}

$Report | export-csv -path h:\GPU-NUMA.csv -NoTypeInformation

[MikeDawson]

I don't know if it will help, but this is a script I made to find information about the GPU usage of our ESXi hosts - perhaps you can use bits and pieces of it  

 

#connect-viserver vcenter.fqdn

$GPUCluster = get-cluster *GPU*
$output=@()
$output2 = @()

$hosts = $GPUCluster | Get-VMHost
$hosts = $hosts | Sort-Object name
foreach($thisHost in $hosts)
{
    write-host
    write-host ESXi Host: $thisHost.Name
    $GPUs = $thisHost.ExtensionData.Config.GraphicsInfo
    #write-host $thisHOst.Name has $GPUs.count $GPUs[0].DeviceName GPUs #assumes all GPUs are the same in the host
    #write-host $GPUs.vm 

    $VMs = $thisHost | Get-VM

    foreach ($thisGPU in $GPUs)
    {            
        write-host " - " -noNewLIne
        write-host "PCI Slot " -ForegroundColor Gray -NoNewline
        write-host $thisGPU.PciId -NoNewline
        write-host " - " -NoNewline
        write-host $thisGPU.DeviceName -NoNewline -ForegroundColor Green

        $VMsOnThisGPU = $thisGPU.Vm

        if(!$VMsOnThisGPU)
        {
            write-host " - [UNUSED]"
            $object = New-Object PSObject
            Add-Member -InputObject $object -memberType NoteProperty -name HostName -value $thisHost.Name
            Add-Member -InputObject $object -memberType NoteProperty -name CardType -value $thisGPU.DeviceName
            Add-Member -InputObject $object -memberType NoteProperty -name CardId -value $thisGPU.PciId
            Add-Member -InputObject $object -memberType NoteProperty -name VDIName -value '[EMPTY]'
            Add-Member -InputObject $object -memberType NoteProperty -name ProfileType -value ''
            $output+= $object
        }
        else #this GPU is not empty, so find out what "mode" it's in (which profile is in use)
        {
            $vm = $VMs | ? {$_.id -match $VMsOnThisGPU[0].Value} #hacky way to just find the first's backing type, since they should all match if they're on the same GPU
            $vGPUDevice = $VM.ExtensionData.Config.hardware.Device | where {$_.Backing.Vgpu}
            $ProfileType = $vGPUDevice.Backing.Vgpu
            write-host " - Mode:" $ProfileType "- Slots: " -NoNewline
            $whitespaces = 0;
            $text = ""
            $unusedText = ""
            
            if($profileTYpe -like "*8q") #can fit two per GPU
            {
                $FramebufferPerVM = 8
                $whitespaces = (8 * 4) + 6
                #$whitespaces = (8 * 2) + 6
                $text = "---------USED---------"
                $text = "░░▒▒▓▓▓▓▓USED▓▓▓▓▓▒▒░░"
                $text = "=========USED========="
                $text = "=================USED================="
          $unusedText = "=================OPEN================="
            }

            elseif ($ProfileType -like "*2b")  #can fit eight per GPU
            {
                $FramebufferPerVM = 2
                $whitespaces = (2 * 4)
                #$whitespaces = (2 * 2)
                $text = "==USED=="
          $unusedText = "==OPEN=="


            }

            $NumVMsPerGPU = 16 / $FramebufferPerVM
            $tempTally = 0

            for([int]$i = 0; $i -lt $NumVMsPerGPU; $i++)
            {
                if($VMsOnThisGPU.count -gt $tempTally)
                {
                    write-host "[" -nonewline
                    #write-host (" ").PadRight($whitespaces, " ") -NoNewline
                    write-host $text -NoNewline -ForegroundColor blue
                    write-host "]" -NoNewline
                }
                else
                {
                    write-host "[" -nonewline
                    write-host $unusedText -NoNewline -ForegroundColor Gray
                    write-host "]" -NoNewline
                } 
                $tempTally++
            }
            write-host
        }

        foreach($VDI in $VMsOnThisGPU) #correct way to fill out the csv (not for graphical output)
        {
            $vm = $VMs | ? {$_.id -match $vdi.Value}
            $vGPUDevice = $VM.ExtensionData.Config.hardware.Device | where {$_.Backing.Vgpu}
            $ProfileType = $vGPUDevice.Backing.Vgpu
            $object = New-Object PSObject
            Add-Member -InputObject $object -memberType NoteProperty -Name HostName -Value $thisHost.Name
            Add-Member -InputObject $object -memberType NoteProperty -name CardType -value $thisGPU.DeviceName
            Add-Member -InputObject $object -memberType NoteProperty -name CardId -value $thisGPU.PciId
            Add-Member -InputObject $object -memberType NoteProperty -name VDIName -value $vm.Name
            Add-Member -InputObject $object -memberType NoteProperty -name ProfileType -value $ProfileType
            $output+= $object
        }
    }

}

#$output | Sort-Object hostname | FT