Get-View to replace Get-VM

[matthewcdove]

I have a script this forum helped me run, much of it using Get-VM.  This works well, however, doing some research the script takes a very long time to run (about 75 VC's with around 25,000 total VM's).

I am trying to redo this with a Get-View as many say this only pulls some information vs. everything about the VM and can run in a much smaller time window.  One issue with the below script is teh output duplicates all VM names for each VC, so there is an incorrect loop in there with respect to that part.

What I am trying to get is:  For each vCenter in the text file, an output of:  vCenter, Cluster, VM Name, Power State, Allocated memory, Used Memory

Here is my script:

$vCenters = Get-content 'C:\VMInventory\msat_vclist.txt'

Foreach ($vCenter in $vCenters)

{

Connect-VIServer -Server $vCenter –User username –Password password

}

$Data = @()

Foreach ($VIServer in $Global:DefaultVIServers)

{

$VMs = Get-VM

      Foreach ($VM in $VMs)

          {

$NewInfo = ""|select vCenter, Cluster, Name, PowerState, MemoryMB, HostMemoryUsage

$NewInfo.vCenter = $VIServer.Name

$NewInfo.Cluster = (Get-Cluster -VM $VM).name

$NewINfo.Name = $VM.Name

$NewInfo.PowerState = $VM.Powerstate

$NewInfo.MemoryMB = $VM.MemoryMB

$NewInfo.HostMemoryUsage = $VM.ExtensionData.Summary.QuickStats.HostMemoryUsage

$Data += $NewInfo

          }

}

$Data | Export-csv –path "C:\VMInventory\MSAT_VM_Memory.csv"

Disconnect-VIServer * -Confirm:$false

[Tuple]

Hi Matt,

Here is a partial answer for you:

1) Part 1

# Get-View:

$VMs =Get-view -viewtype virtualmachine

#Update View info with VM host

$VMs.UpdateViewData("Runtime.Host.Name")

This will return a collection of VM view information that you can run in a for Loop.

ForEach ($VM in $VMs)

{

$NewINfo.Name = $VM.Name

$NewInfo.PowerState = $VM.runtime.powerstate

$NewInfo.MemoryMB =  $vm.config.Hardware.MemoryMB

$NewInfo.HostMemoryUsage = $vm.summary.quickstats.hostMemoryUsage

}

2) Getting Cluster Name:

http://psvmware.wordpress.com/tag/get-cluster-name-for-vms-in-powercli/

3) If you are running that many VC and VMs , you may want consider running several scripts in parallel.

One per VCenter . Then gather each of these subreports in one large one.

Note: In something this large, you would probably want to store this information into a larger repository such as a database. You would also want to tweak your script(s) so that you can "cache" as much of the results as possible into collections of objects so you can later search for this information instead of using several "Get"

commands to get the same info.

For instance:

Think of this ratio

Many VMs -> one Host

Many Hosts -> One Cluster

Many Clusters -> VCenter

Many PortGroups -> One VSwitch / DVSwitch

Many Hosts -> Storage

[matthewcdove]

When running the Get-VM script, I broke the VC's into groups of 5 or so.  A longer range project will be to send this directly to a DB.  Currenlty, we need a single snapshot to validate what a VSPP appliance is telling us and compare.  This is a good start.  I'll work on getting the other pieces today.  My biggest issue is where to find all of the properties available.  The information on the cmdlet Get-View is not very complete and am having trouble finding what options there are for me to report from.

[Tuple]

Hi Matt,

I totally agree. The documentation that I have found from VMware on Get-view is very sparse.

I am still finding out interesting things about the VMware API and how it's mapped out.

(Please note, I am making this explanation so that others that might not be as familiar are also able to understand how it works :-)

I would suggest prototyping your get-view by starting with just 1 VM as a test to get the information you need.

Replace <name of VM> with an actual name of a VM in your environment.

1) Open PowerCli

2) Connect to one VCenter

3) Use get-view to retrieve information of 1 VM in your environment.

$VM = Get-view -viewtype virtualmachine -filter @{'name' = '<name of VM>'}

4)The results of that call return an "Object" and allow you access information and do things with it.

  The easiest way to get used to the structure is to think the Object like a Folder (Directory) in a computer.

 

   Inside a folder there could be:

   1) Files

   2) Subfolders

   3) Executables

   4) Shortcuts to other folders

   5) Hidden files and folders

  

   Inside an Object there are things very similar:

  1) Attributes (think files )

  2) Other Objects (think subfolder)

  3) Methods (think Executables or powershell scripts)

  4) Linked Views (think shortcuts)

  5) References to other objects that haven't called (think hidden files) 

Just to get used to how things are organized, start looking around as if it were a folder you were looking at in a command prompt only instead of a "\" or "/" use a "."

Example:

$VM is the Virtual Machine view returned by :

$VM = Get-view -viewtype virtualmachine -filter @{'name' = '<name of VM>'}

The Name of that VM would be:

$VM.Name

The Memory setting of that VM would be in the following:

$VM.config.Hardware.MemoryMB

When you start getting into using the UpdateViewData command , it gets a little more complicated.

It's like putting "Shortcut Folder" into the Object "Folder" that points to other things.

Example:

In order to get the name of ESXi Host that this VM is running on you would need to use the UpdateView Method to "Create a shortcut" to the ESXi Host properties.

$VM.UpdateViewData("Runtime.Host.Name")

The to access it ,you go through the LinkedView "Shortcut" to it put in $VM.Runtime.

The path to Host name would be:

$VM.Runtime.LinkedView.Host.Name

I hope that helps out a bit :-)

[mattboren]

Hello, matthewcdove-

Focusing on your statement of:

What I am trying to get is:  For each vCenter in the text file, an output of:  vCenter, Cluster, VM Name, Power State, Allocated memory, Used Memory

...you have the pieces in just the right order to make an optimal data collection call:  vCenter -> Cluster -> VMs.  Somewhat like TupleSpace was suggesting about doing as few "Get" calls as possible, if you get inventory items in this order, you can succeed with just a bare minimum of Get calls.  Like:

## txt file of vCenter names, one per line
$strVCNamesFilespec = "c:\temp\vCenterNames.txt"

## get the vCenter server names from the given txt file
Get-Content $strVCNamesFilespec | %{
    $strThisVCName = $_; Write-Verbose -Verbose "working on vCenter '$strThisVCName'"
    ## connect to this vCenter
    $oThisVIServer = Connect-VIServer -Server $strThisVCName
    ## get the cluster Views in this vCenter
    Get-View -ViewType ClusterComputeResource -Property Name | %{
        $viewThisCluster = $_; Write-Verbose -Verbose "working on cluster '$($viewThisCluster.Name)'"
        ## get the VirtualMachine Views in this cluster
        Get-View -ViewType VirtualMachine -Property Name,Runtime.PowerState,Config.Hardware.MemoryMB,Summary.QuickStats.HostMemoryUsage -SearchRoot $viewThisCluster.MoRef | %{
            ## return a new object with info about this VirtualMachine
            New-Object -Type PSObject -Property ([ordered]@{
                vCenter = $strThisVCName
                Cluster = $viewThisCluster.Name
                Name = $_.Name
                PowerState = $_.Runtime.PowerState
                MemoryMB = $_.Config.Hardware.MemoryMB
                HostMemoryUsage = $_.Summary.QuickStats.HostMemoryUsage
            }) ## end new-object
        } ## end foreach-object
    } ## end foreach-object

    ## disconnect from this vCenter
    Disconnect-VIServer $oThisVIServer -Confirm:$false
} ## end foreach-object

This uses the -SearchRoot parameter to Get-View to get all of the VirtualMachine View objects in the given Cluster.  By doing so, you eliminate the Get call that you were using for each VM to get the VM's cluster (eliminate almost all of the 25,000 Get-Cluster calls that you were making, one per VM).  So, per vCenter, it is now just:

• one (1) Get-View call to get all of the cluster Views
• one (1) Get-View call per cluster to get all of the VMs in that cluster

Using the -Property parameter is where the huge time/memory savings come in:  this allows one to have the Get-View call return _only_ the desired properties for the View objects, instead of all of the properties of them.

The UpdateViewData() method of View objects mentioned in one reply is definitely a useful item at the right times.  However, we can keep it a bit simpler in this case by getting the inventory items in the right order.

As for the docs for Get-View -- I find them to be mostly sufficient.  Understanding what type of objects (View objects) are returned is a separate topic, and that is where the vSphere API documentation comes into play.  The "options to report from" become "anything available in the API", which is what makes Get-View and the objects returned so useful/powerful.

And, since sifting through the API documents to trace down through property trees can become confusing/noodle-y, another helpful tool is something like the script editor from PowerGUI that has a Variables inspector.  So, you can get a View object, and then interactively inspect its properties in the Variables pane, like traversing through a folder structure.

Anyway, how does that do for your reporting -- any faster?  And, you can, of course, add your data export call to the end of the code so as to capture the data in [wherever you like] (say, Export-Csv).

[Tuple]

Well written piece of code Matt :-)

I would really like to "Understanding what type of objects (View objects) are returned" to better understand more about the inner-workings of the vSphere API.
Experience has always taught me that more one deeply understands how things work, the easier it becomes work with things.

I am really interested in finding the right Get-View and vSphere API documentation.

I seem to be running into dead ends in my search through the VMware documentation.

If you could point me in the right direction, I would really appreciate it.

Question: Since there are so many VCenters involved in this environment, I am thinking we could take your initial code, modify it slightly and encapsulate it in a Workflow with  Parameters such as VCenter Name  that could be passed to it. Then in the workflow we could run a "foreach -parallel" loop for each VCenter. This allow the calls to each VCenter to run in Parallel not sequentially .

Workflow

1) Read text file of VCenter info

2) Put a modified version of your script in a script block passing VCenter info as a Parameter

3) Encapsulate that script block in a "Foreach -parallel" loop.

The issue being that I think that workflows run as a separate thread as separate process . Inside the workflow , it may need to load the VMware plugin  .

This may have a significant performance hit.

Just a thought.

-Frank :-)

[mattboren]

Thanks, Frank.  Yes, getting the handle on the things with which you are working as returned by Get-View is the ticket.

And, if you are hitting dead ends in the vSphere API documentation, you might not be seeing that:  the vSphere API documentation _is_ the spot.  When you have the View object of a HostSystem, you refer to the API docs to get the properties and "operations" (methods) of such an object.

You can read about _about_ the API docs themselves at VMware vSphere API Reference, which talks about the data structures available in the API.  And, importantly, there is a piece that gives a Managed Object Types overview.  This helps explain about the managed objects, which then helps one understand more about the objects returned by Get-View (references to those managed objects).

As for your question about PowerShell Workflows -- I think that running those things less serially is a great idea.  I do not, however, have a good understanding of PS Workflows, but am definitely intrigued by their ability to do things in parallel.  I agree with you about the separate threads/processes potential problem, and further:  Workflow script blocks are PowerShell "activities", which, from what I do understand, do not necessarily behave the same as a "regular" PowerShell script block.  So, there is some investigating to do in order to help show if/how Workflows can be applied in such a case.

Are you about to whip out some Workflow examples, so that we can all take advantage? :smileysilly:  And, let us know what are your next questions -- this community is here to help, for sure!

[Tuple]

I should be able to do some testing with a workflow sometime next week as long I can get a little free time from the network project I am working on (setting up the network for a new branch can be quite time consuming)

I'll see what I can do :-)

p.s

Thanks again for all your help and useful links :-)

Thanks,

Frank :-)

[Ilnur_Zaripov]

Try this

$vCenters = Get-content 'C:\VMInventory\msat_vclist.txt'

$Data = @()

Foreach ($vCenter in $vCenters)

    {

    Connect-VIServer -Server $vCenter –User username –Password password

    $Hosts = Get-VMHost -Location $vCenter | select -Exp Name

        Foreach ($Host in $Hosts)

            {

            $VMs = Get-VM -Location $Host | select -Exp Name

                Foreach ($VM in $VMs)

                    {

                    $NewInfo = "" | select vCenter, Cluster, Name, PowerState, MemoryMB, HostMemoryUsage

                    $NewInfo.vCenter = $VIServer.Name

                    $NewInfo.Cluster = (Get-Cluster -VM $VM).name

                    $NewINfo.Name = $VM.Name

                    $NewInfo.PowerState = $VM.Powerstate

                    $NewInfo.MemoryMB = $VM.MemoryMB

                    $NewInfo.HostMemoryUsage = $VM.ExtensionData.Summary.QuickStats.HostMemoryUsage

                    $Data += $NewInfo

                    }

            }

}

$Data | Export-csv –path "C:\PowerCLI\MSAT_VM_Memory.csv"

Disconnect-VIServer * -Confirm:$false