PowerCLI

 View Only
Expand all | Collapse all

Multithreading PowerCLI with RunspacePool

  • 1.  Multithreading PowerCLI with RunspacePool

    Posted Jun 12, 2015 10:48 AM

    I am trying to create a script that collect a lot of info from VM's and also set info if needed. That is not my problem. The problem is that there is alot of VM's and it takes about 5-8 sec pr VM. I have create a small sample script to run a process on several VM's at the same time, but it fails in mysterious ways :smileycry: If I only allow 1 thread everything is behaving as expected. More than one thread, it fails with messages that states the PowerCLI cmdlet are not known "The term 'Set-PowerCLIConfiguration' is not recognized as the name of a cmdlet"

    I am using PowerCLI 6

    Any ideas on whats wrong?

    Below is my sample code (some of it is from Google :smileywink:)

    # The per VM codeblock sample

    $scriptBlock = {

      param($VMName, $vcenter, $session)

      Write-Host "VMThread - Start for: $($VMName)"

      Import-Module VMware.VimAutomation.Core -Global

      Write-Host "VMThread - Modules loaded: $(Get-Module)"

      $Temp = Set-PowerCLIConfiguration -InvalidCertificateAction Ignore -Scope Session -Confirm:$false

      $myVI = Connect-VIServer -Server $vcenter -Session $session

      $Myvm = Get-VM -Name $VMName

      if($Myvm)

      {

      Write-Host "VMThread - Getting VM info: $($Myvm.Name)"

      $newInfo = New-Object PSObject

      $newInfo | Add-Member -Type noteproperty -Name VMName -Value $Myvm.Name

      $newInfo | Add-Member -Type noteproperty -Name MemMB -Value $Myvm.MemoryMB

      $newInfo

      Start-Sleep -Seconds 3

      }

      else

      {

      Write-Host "VMThread - No info for VM: $($VMName) - Modules: $(Get-Module)" -ForegroundColor Red

      }

      Write-Host "VMThread - Done: $($VMName) - Error Count: $($Error.Count) - $Error"

    }

    #Start the threads and wait for result

    function RunTest{

      param

      (

      $VMs,

      $Throttle = 1 #threads

      )

      process

      {

      $sessionstate = [System.Management.Automation.Runspaces.InitialSessionState]::CreateDefault()

      $sessionstate.ImportPSModule("VMware.VimAutomation.Core")

      $RunspacePool = [runspacefactory]::CreateRunspacePool(1, $Throttle, $sessionstate, $Host)

      $RunspacePool.Open()

      $Jobs = @()

      Write-Host "Starting threads.."

      foreach ($oneVM in $VMs)

      {

        $Job = [powershell]::Create().AddScript($ScriptBlock).AddArgument($oneVM.Name).AddArgument($global:DefaultVIServer.Name).AddArgument($global:DefaultVIServer.SessionSecret)

        $Job.RunspacePool = $RunspacePool

        $Jobs += New-Object PSObject -Property @{RunVM = $_.Name; Pipe = $Job; Result = $Job.BeginInvoke()}

      }

     

      Write-Host "Start waiting.."

      Do

      {

        Start-Sleep -Seconds 1

      }

      While ( $Jobs.Result.IsCompleted -contains $false)

      Write-Host "All jobs completed!"

     

      $Results = @()

      ForEach ($Job in $Jobs)

      {

      $Results += $Job.Pipe.EndInvoke($Job.Result)

      }

      $RunspacePool.Close()

      Write-Host "`n`nReturning $($Results.Count) objects..." -ForegroundColor Cyan

      $Results

      }

    }

    # Logon to vcenter

    $user = Get-Credential domain\user

    $vi = connect-viserver myvCenter -Credential $user

    $AllVM = Get-VM                     # Return all my VM's, 2 in my test vcenter

    RunTest -VMs $AllVM -Throttle 1     # Runs OK, returns one object for each VM

    RunTest -VMs $AllVM -Throttle 2     # Fails



  • 2.  RE: Multithreading PowerCLI with RunspacePool

    Posted Jun 12, 2015 11:40 AM

    Can you display $PSModulePath in each thread ?

    Is the content correct ?

    Or try doing a 'Get-Module -ListAvailable' in each thread.



  • 3.  RE: Multithreading PowerCLI with RunspacePool

    Posted Jun 23, 2015 09:13 AM

    Hi

    Sorry for the slow response, but i did not get a notification mail :smileyplain:

    Outputs from the 2 commands are similar on all threads. It looks like this:

    VMThread - PSModulePath: C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules\;C:\Program Files\Microsoft Monitoring Agent

    \Agent\PowerShell\;C:\Program Files (x86)\Microsoft SQL Server\110\Tools\PowerShell\Modules\

    VMThread - Get-Module: VMware.VimAutomation.Cis.Core VMware.VimAutomation.Cloud VMware.VimAutomation.Core VMware.VimAuto

    mation.HA VMware.VimAutomation.PCloud VMware.VimAutomation.SDK VMware.VimAutomation.Storage VMware.VimAutomation.Vds App

    Locker Appx BestPractices BitsTransfer BranchCache CimCmdlets DirectAccessClientComponents Dism DnsClient International

    iSCSI IscsiTarget ISE Kds Microsoft.PowerShell.Diagnostics Microsoft.PowerShell.Host Microsoft.PowerShell.Management Mic

    rosoft.PowerShell.Security Microsoft.PowerShell.Utility Microsoft.WSMan.Management MMAgent MsDtc NetAdapter NetConnectio

    n NetEventPacketCapture NetLbfo NetNat NetQos NetSecurity NetSwitchTeam NetTCPIP NetworkConnectivityStatus NetworkTransi

    tion NFS PcsvDevice PKI PrintManagement PSDesiredStateConfiguration PSDiagnostics PSLog PSScheduledJob PSWorkflow PSWork

    flowUtility RemoteDesktop ScheduledTasks SecureBoot ServerCore ServerManager ServerManagerTasks SmbShare SmbWitness Soft

    wareInventoryLogging StartScreen Storage TLS TroubleshootingPack TrustedPlatformModule UserAccessLogging VpnClient Wdac

    Whea WindowsDeveloperLicense WindowsErrorReporting WindowsSearch Microsoft.MonitoringAgent.PowerShell SQLASCMDLETS SQLPS



  • 4.  RE: Multithreading PowerCLI with RunspacePool

    Posted Aug 07, 2015 11:17 AM

    Seems like module exports cmdlets in one runspace only:

    ModuleType Version    Name                                ExportedCommands                                                                                                                       

    ---------- -------    ----                                ----------------                                                                                                                       

    Manifest   3.1.0.0    Microsoft.PowerShell.Management     {Add-Computer, Add-Content, Checkpoint-Computer, Clear-Content...}                                                                     

    Manifest   3.1.0.0    Microsoft.PowerShell.Utility        {Add-Member, Add-Type, Clear-Variable, Compare-Object...}                                                                              

    Manifest   6.0.0.0    VMware.VimAutomation.Core                                                                                                                                                  

    Manifest   6.0.0.0    VMware.VimAutomation.Sdk       

                                                                                                                                              

    Manifest   3.1.0.0    Microsoft.PowerShell.Management     {Add-Computer, Add-Content, Checkpoint-Computer, Clear-Content...}                                                                     

    Manifest   3.1.0.0    Microsoft.PowerShell.Utility        {Add-Member, Add-Type, Clear-Variable, Compare-Object...}                                                                              

    Manifest   6.0.0.0    VMware.VimAutomation.Core           {Add-PassthroughDevice, Add-VirtualSwitchPhysicalNetworkAdapter, Add-VMHost, Add-VMHostNtpServer...}                                   

    Manifest   6.0.0.0    VMware.VimAutomation.Sdk                                                                                                                    



  • 5.  RE: Multithreading PowerCLI with RunspacePool

    Posted Aug 07, 2015 11:26 AM

    I guess this is a bug and should be passed over to VMware.

    Previous PowerCLI version worked well with RunSpaces



  • 6.  RE: Multithreading PowerCLI with RunspacePool

    Posted Aug 07, 2015 01:37 PM

    I don't think this is really a bug, but I suspect it is due to the fact the current PowerCLI release (6R1) uses a mix of modules and pssnapins.

    In fact this causes some other minor issues as well.

    Can't wait to see a full module release.



  • 7.  RE: Multithreading PowerCLI with RunspacePool

    Posted Jan 26, 2016 12:23 PM

    Hi CNI0 !

    Basically the fastest way to do what you wanted is this one:

    $allVms = Get-View -ViewType VirtualMachine -Property Name,"Config.Hardware","Config.Template" -Filter @{"Config.Template"="False"}

    $allVms | Select @{N="VMName";E={$_.Name}},@{N="MemMB";E={$_.Config.Hardware.MemoryMB}} | Out-GridView

    You'll be impressed with how fast it is, even without runspaces!

    Or, if you really want to use runspaces, then you could use my PS module, which contains functions to work with RunSpaces like with Jobs. You could grab it here: monosoul/basic_functions.psm1 · GitHub

    (readme may be little outdated, byt every function has built-in documentation)

    Here is an example code for what you wanted to do:

    $user = Get-Credential domain\user 

    $vi = connect-viserver myvCenter -Credential $user

    $snapins = @()

    $modules = @()

    $snapins += "VMware.VimAutomation.Core"

    if ((Get-PowerCLIVersion).Major -le 5) {

      $snapins += "VMware.VimAutomation.Vds"

      $snapins += "VMware.VimAutomation.Storage"

    }

    else {

      $modules += "VMware.VimAutomation.Vds"

      $modules += "VMware.VimAutomation.Storage"

    }

    $AllVM = Get-VM

    #basically it's still better to use Get-View as it'd be a lot faster, but i'll stick to your example

    #$AllVM = Get-View -ViewType VirtualMachine -Property Name,"Config.Hardware","Config.Template" -Filter @{"Config.Template"="False"}

    #making synchronized queue out of array

    [System.Collections.Queue]$AllVM = [System.Collections.Queue]::Synchronized( ([System.Collections.Queue]$AllVM) )

    #creatin synchronized array list for results

    $newInfo = [System.Collections.ArrayList]::Synchronized( (New-Object System.Collections.ArrayList) )

    Start-RSJobs -ScriptBlock {

      param(

        $AllVM,

        $newInfo

      )

      while ($AllVM.Count -gt 0) {

        $Myvm = $AllVM.Dequeue()

        $newInfo.Add((New-Object PSObject -Property @{

          VMName = $Myvm.Name

          MemMB = $Myvm.MemoryMB

        }))

      }

    } -snapinsToImport $snapins `

      -modulesToImport $modules `

      -variablesToImport @("DefaultVIServer", "DefaultVIServers") `

      -ArgumentList $AllVM, $newInfo `

    | Wait-RSJob | Remove-RSJob

    $newInfo

    As of modules importing only in first runspace - yeah, I've faced that issue too even with PCLI6R3, and I think it's some kind of bug. But for now I solved it by using snap-in version of "VMware.VimAutomation.Core" (other modules are importing fine in every runspace).



  • 8.  RE: Multithreading PowerCLI with RunspacePool

    Posted Apr 26, 2017 04:32 AM

    I found a solution that allows me to still run multithread using runspaces. Since runspaces aren't threadsafe, you have to run each runspace outofprocess. You don't need to use snapins with this solution.  This thread is unanswered for 2 years.  I guess it's not priority for Vmware but I figured I'll share it as I haven't found a good solution by googling for past 2 days.  Here's what I had to do in my c# code. You just need to make adjustments in your powershell code to do the same by calling the RunspaceFactory.CreateOutOfProcessRunspace() Can you send me your code that you converted using the SDK? That would be helpful in my project too.  Thanks

            public async Task TestPowercli(string name, string vcenterHost) {

                if (string.IsNullOrWhiteSpace(name)) { return; }

                if (string.IsNullOrWhiteSpace(vcenterHost)) { return; }

                instanceName = name.Trim();

                int timeoutMins = 5;

                string script = @"c:\temp\testpowercli\testpowercli.ps1 " + instanceName + " " + vcenterHost + " -verbose";

                PowerShellProcessInstance instance = new PowerShellProcessInstance(new Version(5, 0), null, null, false);

                using (Runspace runspace = RunspaceFactory.CreateOutOfProcessRunspace(new TypeTable(new string[0]), instance)) {

                    PowerShell ps = PowerShell.Create();

                    runspace.Open();

                    ps.Runspace = runspace;

                    ps.AddScript(script);

                    outputCollection.DataAdded += outputCollection_DataAddedPowercli;

                    // the streams (Error, Debug, Progress, etc) are available on the PowerShell instance.

                    // we can review them during or after execution.

                    // we can also be notified when a new item is written to the stream (like this):

                    ps.Streams.Error.DataAdded += Error_DataAddedPowercli;

                    ps.Streams.Verbose.DataAdded += Verbose_DataAddedPowercli;

                    IAsyncResult result = ps.BeginInvoke<PSObject, PSObject>(null, outputCollection);

                    DateTime start = DateTime.Now;

                    while (result.IsCompleted == false) {

                        if ((DateTime.Now - start).TotalMinutes > timeoutMins) {

                            Clients.All.getPowercliMessage(instanceName, "ERROR: Time out exceeded after " + timeoutMins + " minutes");

                            break;

                        }

                        await Task.Delay(1000);

                    }

                }

                Clients.All.getPowercliMessage(instanceName, "Done");

            }



  • 9.  RE: Multithreading PowerCLI with RunspacePool

    Posted Jun 06, 2018 06:37 PM

    This post saved my soul. If you're like me, you maybe needed some deeper separation between the powershell runspaces and wanted to do multithreaded copies of files to remote datastores. The problem with the current powercli implementation is that it maps psdrives in a local scope, but makes the defaultviservers variable in a global scope. My recommendation to VMware is that the next version of powercli includes an option on the "Connect-VIServer" cmdlet to specify a scope of that connection, or at least patch the psdrives for "vmstores" and "vis" to a global scope that can be shared among runspaces.

    I took the liberty of writing it out in powershell. This basically is an alternative to a runspace pool and should work in any powershell version past 3.0:

    $Active_Jobs = @()

    $index = 0

    while ($true) {


       if (($Active_Jobs.count -lt $num_threads) -and $index -lt $some_list.Length) {

       $job = ($some_list[$index])

       Write-Host "Creating job for $job"


       $psinstance = [System.Management.Automation.PowerShell]::create()

       Add-Member -InputObject $psinstance -NotePropertyName "request_stop" -NotePropertyValue "N"

       $psinstance.AddScript($script_block).AddArgument($some_list[$index]) | Out-Null

       $index++

       $runspace = [runspacefactory]::CreateOutOfProcessRunspace([System.Management.Automation.Runspaces.TypeTable]::GetDefaultTypeFiles())

       if([Console]::InputEncoding -is [Text.UTF8Encoding] -and [Console]::InputEncoding.GetPreamble().Length -ne 0) {

       [Console]::InputEncoding = New-Object Text.UTF8Encoding $false

      }


       $runspace.Open()

       $psinstance.Runspace = $runspace

       $InputStream = New-Object -Typename System.Management.Automation.PSDataCollection[PSObject]

       $OutputStream = New-Object -Typename System.Management.Automation.PSDataCollection[PSObject]

       $Active_Jobs += New-Object PSObject -Property @{

      Thread = $psinstance

      Handle = $psinstance.BeginInvoke($InputStream,$OutputStream)

      runspace = $runspace

      output = $OutputStream

      }

       Start-Sleep -Seconds 1

      }


       Foreach ($Job in $($Active_Jobs | Where-Object {$_.Thread.request_stop -eq "Y"})) {

       $Job.Thread.Stop()

       Write-Host $Job.output

       $Job.Thread.Dispose()

       $Job.Runspace.Close()

       $Job.Runspace.Dispose()

       $Active_Jobs = $Active_Jobs -ne $Job

      }


       Foreach ($Job in $($Active_Jobs | Where-Object {$_.Handle.IsCompleted -eq $true})) {

       Write-Host $Job.output

       $Job.Thread.EndInvoke($Job.Handle) 2>$null

       $Job.Thread.Dispose()

       $Job.Runspace.Close()

       $Job.Runspace.Dispose()

       $Active_Jobs = $Active_Jobs -ne $Job

      }


       if ($Active_Jobs.count -eq 0 -and $index -eq $some_list.Length){

       break

      }


       Start-Sleep -Seconds 2

    }

    Write-Host "Jobs completed!"

    The above code is provided as-is! Do not expect any kind of support from me on it.



  • 10.  RE: Multithreading PowerCLI with RunspacePool

    Posted Jun 06, 2018 08:58 PM

    Nice one.
    And yes, I support your suggestion on the scope option.
    Why don't you raise that as an idea in PowerCLI Ideas?



  • 11.  RE: Multithreading PowerCLI with RunspacePool

    Posted Sep 07, 2017 04:04 AM

    Hi monosoul,

    Thanks so much for sharing your code.  I have changed it slightly for my purposes and turned it into a module on github.  The objective was to provide a simple framework for people to customize and add the things they are interested in returning from ESX (using Runspaces of course).

    It seems to be working fine with the latest PowerCLI (tested on 6.5.2).   I am open to ways to make it better.

    GitHub - vmkdaily/EsxRunspace: Connects to one or more VMware ESX hosts using PowerShell Runspace jobs, and returns a re…

    -Mike