Running esxcli commands on all hosts in a cluster

[TheVMinator]

I would like to run a set of esxcli commands on all hosts in a cluster using powercli, and then report back the results.

For each host, I need to get each vmnic in order, and run the commands on each vmnic.  Then I need to iterate to the next host.

For example, how would I run these esxcli commands on each pnic in my cluster and report back a table listing each host and the results of the commands on each pnic of the host?

esxcli network nic vlan stats set -e true -n vmnic0

esxcli network nic vlan stats get -n vmnic0

esxcli network nic vlan stats set -e false -n vmnic0

Thanks!

[LucD]

Try something like this

Get-CLuster -Name MyCluster | Get-VMHost | %{

    $esx = $_

    $esxcli = Get-EsxCli -VMHost $esx

    $esxcli.network.nic.list() | where {$_.Name -match "vmnic"} | %{

        $nic = $_

        Try {

            $esxcli.network.nic.vlan.stats.set($true,$nic.Name)

            $esxcli.network.nic.vlan.stats.get($nic.Name) | %{

                New-Object PSObject -Property @{

                    VMHost = $esx.Name

                    NIC = $nic.Name

                    VLANID = $_.VLANID

                    Received = $_.Packetsreceived

                    Sent = $_.Packetssent

                }

            }

            $esxcli.network.nic.vlan.stats.set($false,$nic.Name)

        }

        Catch {

            Write-Warning "$($esx.Name) - $($nic.name) not connected to a switch"

        }

    }

}

[TheVMinator]

The script seems to look good but the try catch is always moving to the catch clause even when the nics are connected to a switch.  What permissions do I need for this to execute?

[LucD]

Are you using distributed switches by any chance ?

[LucD]

I see the same behavior on NICs connected to a distributed switch.

It seems the 'get' comes too soon (before the 'set' is completed) for NICs that are on distributed switches.

When I put a sleep between the 2 lines, it works for me.

Get-Cluster -Name MyCluster | Get-VMHost | %{

    $esx = $_

    $esxcli = Get-EsxCli -VMHost $esx

    $esxcli.network.nic.list() | where {$_.Name -match "vmnic"} | %{

        $nic = $_

        Try {

            $esxcli.network.nic.vlan.stats.set($true,$nic.Name)

            sleep 1

            $esxcli.network.nic.vlan.stats.get($nic.Name) | %{

                New-Object PSObject -Property @{

                    VMHost = $esx.Name

                    NIC = $nic.Name

                    VLANID = $_.VLANID

                    Received = $_.Packetsreceived

                    Sent = $_.Packetssent

                }

            }

            $esxcli.network.nic.vlan.stats.set($false,$nic.Name)

        }

        Catch {

            Write-Warning "$($esx.Name) - $($nic.name) not connected to a switch"

        }

    }

}

[TheVMinator]

OK great that works now.  One more thing - can I modify the layout in such a way to be able to compare the vlan IDs seen on each NIC and insure they are consistent across a cluster?

[TheVMinator]

(and export it to csv ? the current report doesn't format very well when I try to export) thanks again!

[LucD]

Try the following, it assumes a couple of things:

• the same vmnicx is used for the same portgroup(s) on each ESXi node in the cluster
• the script only states if it sees the same VLANs on different ESXi nodes, it will not list what VLANs are missing
• the first returned ESXi node is used as the reference. It compares all the other nodes in the cluster with this one.

$tab = @()

Get-Cluster -Name MyCluster | Get-VMHost | %{

    $esx = $_

    $esxcli = Get-EsxCli -VMHost $esx

    $esxcli.network.nic.list() | where {$_.Name -match "vmnic"} | %{

        $nic = $_

        $obj = New-Object PSObject -Property @{

            VMHost= $esx.Name

            NIC = $nic.Name

            VLANIDs = @()

        }

        Try {

            $esxcli.network.nic.vlan.stats.set($true,$nic.Name) | Out-Null

            sleep 1

            $esxcli.network.nic.vlan.stats.get($nic.Name) | %{

                $obj.VLANIDs += $_.VLANID

            }

            $esxcli.network.nic.vlan.stats.set($false,$nic.Name) | Out-Null

        }

        Catch {

            Write-Warning "$($esx.Name) - $($nic.name) not connected to a switch"

        }

        $obj.VLANIDs = $obj.VLANIDs | Sort-Object

        $tab += $obj

    }

}

foreach($nic in ($tab | Group-Object -Property NIC)){

    if($nic.Group[0].VLANIDs){

        $nic.Group[1..($nic.Group.Count - 1)] | %{

            $result = Compare-Object -ReferenceObject $nic.Group[0].VLANIDs -DifferenceObject $_.VLANIDs

            $result

            if($result){

                Write-Output "$($nic.Group[0].NIC) on $($nic.Group[0].VMHost) and $($_.VMHost) has NOT the same VLANs"

            }

            else{

                Write-Output "$($nic.Group[0].NIC) on $($nic.Group[0].VMHost) and $($_.VMHost) has the same VLANs"

            }

        }

    }

}

Be aware that this a early draft of the script, there is no decent error checking present (yet)

[TheVMinator]

Ok that's great - one problem though - the situation here is that there isn't one "known good" host.  We don't really know what vlans are or should be trunked - we want to use the script as a "discovery" to determine what are ALL the vlans that are present on any NIC on the entire cluster.  From that we can go back and determine which vlans are missing from which trunks.  So we would like to:

1.  Identify the set of ALL vlans

2.  List for each host at a host level which of them they have - we can look at this at the host level for now rather than at the vnic level

3.  Determine which vlans are missing from which hosts.

Is that possible?

Thanks again!

[LucD]

The script already contains the VLANs per host per vmnic, they are in the $tab variable.

If you want to compare on the host level, the same question remains, which is the reference host ?

[TheVMinator]

Thanks for the input. Perhaps I need to approach this from a different angle.   Given these assumptions:

1.  If a VLAN is trunked to any host in the cluster, it should be trunked to all hosts.

2.  It isn't known if ANY of the hosts can function as a reference - neglect of VLAN trunking could have taken place on any or all hosts.

Probably I need to do this:

1.  Get a list that lists each vlan present on the cluster - one line per VLAN.  I have to compile this list by looking at every esxi host, running the commands, and rolling it up into the list

2.  For each VLAN, list the ESXI hosts that it is trunked to, and the ESXi hosts that it is NOT trunked to.  Then i can go to the network team, and have them trunk the vlans to the hosts and interfaces that they are not yet trunked to.

I do apologize that this changes the original design of the report somewhat. Should we start a different thread for this?

For example:

VLAN ID	Trunked to	Not Trunked To
135	esxi100, esxi101, esxi102	esxi103,esxi104,esx105
150	esxi100, esxi101	esxi102, esxi103, esxi104, esxi105

[LucD]

Try like this

$clusterName = 'MyCluster'

$tab = @{}

$vmhosts = Get-Cluster -Name $clusterName | Get-VMHost

foreach($esx in $vmhosts){

    $esxcli = Get-EsxCli -VMHost $esx

    $esxcli.network.nic.list() | where {$_.Name -match "vmnic"} | %{

        $nic = $_

        Try {

            $esxcli.network.nic.vlan.stats.set($true,$nic.Name) | Out-Null

            sleep 1

            $esxcli.network.nic.vlan.stats.get($nic.Name) | %{

                if($tab.ContainsKey($_.VLANID)){

                    if($tab[$_.VLANID] -notcontains $esx.Name){

                        $tab[$_.VLANID] += $esx.Name

                    }

                }

                else{

                    $tab.Add($_.VLANID,@($esx.Name))

                }

            }

            $esxcli.network.nic.vlan.stats.set($false,$nic.Name) | Out-Null

        }

        Catch {

            Write-Warning "$($esx.Name) - $($nic.name) not connected to a switch"

        }

    }

}

$tab.GetEnumerator() | Sort-Object -Property Name | %{

    $presentIn = $_.Value

    $notPresentIn = @($vmhosts | where {$_.Name -notin $presentIn} | %{$_.Name})

   

    [PSCustomObject]@{

        VLANID = $_.Name

        TrunkedTo = [string]::Join(',',$presentIn)

        NotTrunkedTo = [string]::Join(',',$notPresentIn)

    }

}