Set coredump location on ESXi hosts to the vCenter it resides on

[dbutch1976]

I have serveral different vCenters and I recently discovered that they can be configured as core dump locations by turning the respective service on. I found this which was a good starting point:

https://communities.vmware.com/t5/VMware-PowerCLI-Discussions/Set-Network-Coredump-Settings-with-PowerCLI-Problem/td-p/1232910

These are the specific esxcli commands I need to run on each host:

$esxcli.system.coredump.network.set($null,"vmk0","192.168.22.250",6500)
$esxcli.system.coredump.network.set(1)
$esxcli.system.coredump.network.get()

How can I write the script so that it sets these variables using the IP of the vCenter that the host resides on? I know that I can do this to determine the vcenter name:

Get-VMHost -Name w0617labhyp01.tdlab.ca | select Parent

However I believe that the $esxcli.system.coredump.network.set command is expecting an IP address rather than the FQDN. Is the IP of the vCenter stored anywhere?

 

[LucD]

You could try something like this

 

 

Get-VMHost -PipelineVariable esx |
ForEach-Object -Process {
   $esxcli = Get-EsxCli -VMHost $esx -V2
   Write-Host "Looking at $($esx.Name)"
   Write-Host "`tBefore change"
   $esxcli.system.coredump.network.get.Invoke()

   $vc = ([uri]$esx.ExtensionData.Client.ServiceUrl).Host
   $arguments = $esxcli.system.coredump.network.set.CreateArgs()
   $arguments.interfacename = 'vmk0'
   $arguments.serverip = ([System.Net.Dns]::GetHostAddresses($vc)).IPAddressToString
   $arguments.serverport = 6500
   $esxcli.system.coredump.network.set.Invoke($arguments)

   $arguments = $esxcli.system.coredump.network.set.CreateArgs()
   $arguments.enable = $true
   $esxcli.system.coredump.network.set.Invoke($arguments)

   Write-Host "`tAfter change"
   $esxcli.system.coredump.network.get.Invoke()
}

 

 

[dbutch1976]

I got:

Get-VMHost : Missing an argument for parameter 'Name'. Specify a parameter of type 'System.String[]' and try again.
At line:1 char:12
+ Get-VMHost -Name -PipelineVariable esx
+ ~~~~~
+ CategoryInfo : InvalidArgument: (:) [Get-VMHost], ParameterBindingException
+ FullyQualifiedErrorId : MissingArgument,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetVMHost

 

Just had to drop -name and it worked perfectly, thanks LucD!

Get-VMHost -PipelineVariable esx |
ForEach-Object -Process {
Write-Host "Looking at $($esx.Name)"
Write-Host "`tBefore change"
$esxcli.system.coredump.network.get.Invoke()

$vc = ([uri]$esx.ExtensionData.Client.ServiceUrl).Host
$esxcli = Get-EsxCli -VMHost $esx -V2
$arguments = $esxcli.system.coredump.network.set.CreateArgs()
$arguments.interfacename = 'vmk0'
$arguments.serverip = ([System.Net.Dns]::GetHostAddresses($vc)).IPAddressToString
$arguments.serverport = 6500
$esxcli.system.coredump.network.set.Invoke($arguments)

$arguments = $esxcli.system.coredump.network.set.CreateArgs()
$arguments.enable = $true
$esxcli.system.coredump.network.set.Invoke($arguments)

Write-Host "`tAfter change"
$esxcli.system.coredump.network.get.Invoke()
}

[LucD]

Yes, that was a typo, that Name parameter shouldn't be in there.
I corrected the code above

[dbutch1976]

One last thing that would fully automate this process, do you know how to start the VMware vSphere ESXi Dump Collector service and set the startup type to automatic from within PowerCLI? I can't seem to find any documentation on it.

[LucD]

Afaik there are no PowerCLI cmdlets or tools to do that.
But since the VMware Tools are running on the VCSA, you can use Invoke-VMScript to send the following commands to the VCSA.

vmon-cli -S AUTOMATIC -U netdumper
service-control --start  vmware-netdumper

[dbutch1976]

Hey LucD,

I haven't worked with invoke-vmscript much, just trying to do some basic stuff and it's not working, can you see what I'm doing wrong?

$guestcred = (Get-Credential)
Invoke-VMScript -VM "VCNUC.lebrine.local" -ScriptText "service-control --status" -ScriptType Bash -GuestCredential $guestcred

cmdlet Get-Credential at command pipeline position 1
Supply values for the following parameters:

ScriptOutput
-----------------------------------------------------------------------------------------------------------------------| Traceback (most recent call last):
| File "/usr/bin/service-control", line 12, in <module>
| sys.path.extend(os.environ['VMWARE_PYTHON_PATH'].split(':'))
| File "/usr/lib/python3.7/os.py", line 679, in __getitem__
| raise KeyError(key) from None
| KeyError: 'VMWARE_PYTHON_PATH'
|
-----------------------------------------------------------------------------------------------------------------------

 

Credentials I'm using are root.

[LucD]

The shell that is started via Invoek-VMScript is missing several environment variables.
And for the service-control command that seems to be the VMWARE_PYTHON_PATH variable.

William has a post about how to add these environment variables when using Invoke-VMScript.
See How to remotely run appliancesh & other shell commands on VCSA w/o requiring SSH? (williamlam.com)

[dbutch1976]

Just in case anyone else wants to accomplish the same thing:

Invoke-VMScript -ScriptText "export VMWARE_VAPI_HOME=/usr/lib/vmware-vapi
export VMWARE_RUN_FIRSTBOOTS=/bin/run-firstboot-scripts
export VMWARE_DATA_DIR=/storage
export VMWARE_INSTALL_PARAMETER=/bin/install-parameter
export VMWARE_LOG_DIR=/var/log
export VMWARE_OPENSSL_BIN=/usr/bin/openssl
export VMWARE_TOMCAT=/opt/vmware/vfabric-tc-server-standard/tomcat-7.0.55.A.RELEASE
export VMWARE_RUNTIME_DATA_DIR=/var
export VMWARE_PYTHON_PATH=/usr/lib/vmware/site-packages
export VMWARE_TMP_DIR=/var/tmp/vmware
export VMWARE_PERFCHARTS_COMPONENT=perfcharts
export VMWARE_PYTHON_MODULES_HOME=/usr/lib/vmware/site-packages/cis
export VMWARE_JAVA_WRAPPER=/bin/heapsize_wrapper.sh
export VMWARE_COMMON_JARS=/usr/lib/vmware/common-jars
export VMWARE_TCROOT=/opt/vmware/vfabric-tc-server-standard
export VMWARE_PYTHON_BIN=/opt/vmware/bin/python
export VMWARE_CLOUDVM_RAM_SIZE=/usr/sbin/cloudvm-ram-size
export VMWARE_VAPI_CFG_DIR=/etc/vmware/vmware-vapi
export VMWARE_CFG_DIR=/etc/vmware
vmon-cli -S AUTOMATIC -U netdumper
service-control --start vmware-netdumper
service-control --status
" -vm myvcenter1 -GuestUser root -GuestPassword myp@ass!

*Note, I've exported more envrionmental variables than required which may look like overkill, however this should simply running any other commands.

[dbutch1976]

After using this script to successfully configure the dump collector across my environment (thanks again LucD!), we had our first PSD and the dump was successfully sent to the vCenter. However I can't seem to find instructions anywhere on how to actually retrieve the dump from vCenter in order to send it to VMware for analysis. Although this isn't a PowerCLI question I'd imagine anyone who has set this up is going to be interesting in knowing how to retrieve it, but I can't seem to find instructions anywhere.

Has anyone actually retrieved one of these dumps from vCenter?

[LucD]

The cores are stored in folder /var/core/netdumps on the VCSA.
You can check file /var/log/vmware/netdumper/netdumper.log to check which core dump for which ESXi node was stored where.

[RoyIW]

Hi, just to say works with one slight change.
The 'before' commands run against the previous host.
Needs to move down after

$esxcli = Get-EsxCli -VMHost $esx -V2

 

[LucD]

You are right, I updated the code above.