Using Aria Automation to Deploy Windows Server using Cloudbase-Init

[pizzle85]

I'm pretty late to the party but I found much less useful information on this topic than I expected.  This is my how-to use Aria Automation to deploy Windows Server using Cloudbase-init

Overview

This guide assumes you already have vSphere and Aria Automation installed and configured.

We will be creating an Aria Automation Assembler Template that will clone a sysprepped Windows Server VM with Cloudbase-Init installed to bootstrap the OS. The VM will be on the network and joined to Active Directory.

The process will look something like:

1. User requests a VM in Aria Automation
2. An Event Broker subscription creates an AD object in a specified OU
3. An Event Broker subscription sets the Cloudbase-Init metadata and userdata advanced properties on the VM.
4. Aria Automation requests the VM be created in vCenter
5. The VM boots up and runs sysprep which then asks for Cloudbase-Init to run.
6. Cloudbase-Init runs the unattend conf file using the provided metadata to rename the computer and configure the NIC
7. Sysprep finishes and the VM is rebooted
8. Cloudbase-Init runs the conf file to execute a script to join the computer to the domain.
9. The VM reboots
10. User requests the VM to be destroyed
11. An Event Broker subscription deletes the AD object for the computer

Cloudbase-init

Cloudbase-Init is installed in the template and executes during sysprep using the cloudbase-init-unattend.conf file and at all subsequent boots using the cloudbase-init.conf file.

Cloudbase-Init can get data from VMware two ways:

1. OVF – This is the default method for Aria Automation and is available anywhere you can use an OVF. The metadata and userdata are stored in an OVF and mounted to the VMs virtual CD-ROM.
2. VMware GuestInfo Service – This method stores the metadata and userdata as advanced properties of the VM using gzip base64 encoded strings.

We are going to use the VMware GuestInfo Service in this example.

In this example we will be doing the following to the OS:

1. Sysprep
   1. Rename the computer
   2. Configure the NIC with a static IP address
2. First Boot
   1. Join the VM to an Active Directory domain

Active Directory

Create a new AD user that can be used to create and destroy computer objects and join computers to the domain.

Add the user to an OU with the following advanced permissions in addition to the defaults

• Create computer objects
• Delete computer objects

Join Domain Powershell Script

Create the script and place it in the Cloudbase-init LocalScripts directory of the template server as described in step 5 of the "Virtual Machine Template" section. If you copy the script from another server, be sure to unblock the file so it can be executed.

try {
    $cloud_init_dir = "$env:programfiles\cloudbase solutions\cloudbase-init"
    $log = $cloud_init_dir + "\log\join_domain.log"
    New-Item $log -ErrorAction SilentlyContinue
    write-output "Starting Join Domain Script" | Out-File $log -Append

    write-output "installing YAML Module" | Out-File $log -Append
    install-packageprovider -name nuget -force
    install-module -name powershell-yaml -force
    import-module powershell-yaml -force
    write-output "installed YAML Module" | Out-File $log -Append

    write-output "Getting cloudbase-init metadata" | Out-File $log -Append
    cd "$env:programfiles\vmware\vmware tools"
    $metadata = & .\rpctool.exe "info-get guestinfo.metadata"
    $bytes = [System.Convert]::FromBase64String($metadata)
    $memory_stream = New-Object System.IO.MemoryStream(,$bytes)
    $gzip_stream = New-Object System.IO.Compression.GzipStream($memory_stream, [System.IO.Compression.CompressionMode]::Decompress)
    $stream_reader = New-Object System.IO.StreamReader($gzip_stream)
    $decompressed_string = $stream_reader.ReadToEnd()
    write-output "Got cloudbase-init metadata" | Out-File $log -Append

    $yaml = convertfrom-yaml $decompressed_string
    $hostname = $yaml.hostname
    write-output "Got hostname from metadata: $hostname" | Out-File $log -Append
    write-output "Computername: $env:computername"

    if ($env:computername -ne $hostname) {
        write-output "Computer name has not been updated yet, exiting and rebooting" | Out-File $log -Append
        $runs = (select-string -path $log -pattern 'Computer name has not been updated yet, exiting and rebooting').count
        if ($runs -lt 3) {
            exit 1003
        }
        else {
            throw "Computer name has not been updated after 3 runs, exiting"
        }
    }
    else {
        write-output "Computer name has been updated to $env:computername, joining to domain" | Out-File $log -Append
        $creds = new-object system.management.automation.pscredential ('<username>@<domain>', (convertto-securestring '<password>' -asplaintext -force))
        add-computer -DomainName '<domain>' -Credential $creds -Force
        write-output "Joined to domain" | Out-File $log -Append
        remove-item "$env:programfiles\cloudbase solutions\cloudbase-init\localscripts\join_domain.ps1" -force
        write-output "Removed join_domain.ps1" | Out-File $log -Append
        write-output "Set Execution Policy to restricted" | Out-File $log -Append
        write-output "Exiting with code 1001" | Out-File $log -Append
        exit 1001
    }
}
catch {
    Write-Output "Error: $_" | Out-File $log -Append
    write-output "Removing join_domain.ps1" | Out-File $log -Append
    remove-item "$env:programfiles\cloudbase solutions\cloudbase-init\localscripts\join_domain.ps1" -force
    write-output "Set Execution Policy to restricted" | Out-File $log -Append
}

Virtual Machine Template

1. Create a Virtual machine and manually install Windows Server (or use your favorite machine image creation tool, such as packer)
   1. Install Cloudbase-Init
      1. Configuration Options
         1. Username
            1. Administrator
   2. Close the installer without running sysprep
   3. Configure the cloudbase-init-unattend.conf file.
      1. Add the following lines to use the VMware GuestInfo Service to get the metadata and userdata then use the SetHostNamePlugin and NetworkConfigPlugin.
         1. metadata_services= cloudbaseinit.metadata.services.vmwareguestintoservice.VMwareGuestInfoService
         2. plugins=cloudbaseinit.plugins.common.sethostname.SetHostNamePlugin,cloudbaseinit.plugins.common.networkconfig.NetworkConfigPlugin
   4. Configure the cloudbase-init.conf file.
      1. Add the following lines to use the VMware GuestInfo Service to get the metadata and userdata then use the LocalScriptsPlugin to run a powershell script to join the computer to the domain
         1. metadata_services= cloudbaseinit.metadata.services.vmwareguestintoservice.VMwareGuestInfoService
         2. plugins=cloudbaseinit.plugins.common.localscripts.LocalScriptsPlugin
   5. Copy the join domain powershell script to the Cloudbase-Init LocalScripts directory
      1. Make sure to unblock the file so it can be executed.
   6. Delete any log files in the Cloudbase-Init log directory
   7. Run sysprep using the Cloudbase-Init unattend.xml file
      1. You can run these commands anytime you need to update the template.
   8. 1.      Convert the VM to a template

Aria Automation

Orchestrator

Create Aria Orchestrator workflows to create and delete AD computer objects.

Create AD Computer Object

• inputs
  •  inputProperties
    • type: Properties
    • direction: input
• variables
  • domainName
    • value: <domain_name>
    • type: string
  • ou
    • type: AD:OrganizationalUnit
  • computerName
    • type: string
• schema
  • scriptable task
    • inputs
      • inputProperties
    • outputs
      • computerName
      • out
    • script
      • var cps = inputProperties.get('customProperties')

        
        

        computerName = cps.get('hostname')

        
        

        var ou_name = JSON.parse(cps.get('ou')).label

        var ou_dn = JSON.parse(cps.get('ou')).id.split('#')[7]

        ous = ActiveDirectory.searchExactMatch('OrganizationalUnit', ou_name)

        for each (ou_attr in ous) {

            if (ou_attr.distinguishedName == ou_dn) {

                ou = ou_attr

            }

        }

        if (!ou) {

            throw 'no ou found with dn ' + ou_dn

        }
  • workflow
    • Create a computer in an organizational unit
      • inputs
        • ou: ou
        • computerName: computerName
        • domainName: domainName
  • END

Delete AD Computer Object

• inputs
  • inputProperties
    • type: Properties
• variables
  • computer
    • type: AD:ComputerAD
• schema
  • scriptable task
    • inputs
      • inputProperties
    • outputs
      • computer
    • script
      • var cps = inputProperties.get('customProperties')

        var hostname = cps.get('hostname')

        var computer = ActiveDirectory.searchExactMatch('ComputerAD', hostname, 1)[0]
  • decision
    • inputs
      • computer
    • script
      •  return computer != null
        • return false
          • END
        • return true
          • workflow
            • Destroy a computer
              • inputs
                • computer
          • END

Cloud Account

Sync the Cloud Account images for the cloud account you created the template in.

Image Mapping

Create a new image mapping that points to the newly created template

Networks

Add a tag to a network so that it can be referenced in the template

Storage

Add a tag to a datastore or datastore cluster so that it can be referenced in the template

Template

Create a new template.

formatVersion: 1
  resources:
    Cloud_vSphere_Network_1:
      type: Cloud.vSphere.Network
      properties:
        constraints:
          - tag: <network_tag>
    Cloud_vSphere_Machine_1:
      type: Cloud.vSphere.Machine
     properties:
       name: ${env.deploymentName}
       hostname: ${split(env.deploymentName, '.')[0]}
       image: <image_mapping>
       cpuCount: 2
       totalMemoryMB: 4096
       ip: 192.168.1.2
       cidr: 24
       gateway: 192.168.1.1
       constraints:
         - tag: <storage_tag>
       networks:
         - network: ${resource.Cloud_vSphere_Network_1.id}
       ou: <OU_DN>  

Extensibility

Actions

Create actions

make cloudbase-init guestinfo data

python

import requests
import json
import gzip
import base64
    
def handler(context, inputs):
    outputs = {}
    cps = inputs['customProperties']
    vm_name = cps['hostname']
   
   metadata_string = f'''
instance-id: {vm_name}
local-hostname: {vm_name}
hostname: {vm_name}
network:
  version: 2
  ethernets:
    Ethernet0:
      match:
        macaddress: "{inputs['macAddresses'][0][0]}"
      addresses: [{cps['ip']}/{cps['cidr']}]
      gateway4: {cps['gateway']}
      nameservers:
        addresses: [{inputs['ad_nameserver_1']}, {inputs['ad_nameserver_2']}]
'''
    print(metadata_string)

    def base64_encode_gzip(data):
        """Encodes data with gzip and then Base64."""
        compressed_data = gzip.compress(data.encode())
        encoded_data = base64.b64encode(compressed_data)
        return encoded_data.decode()
     
    outputs['metadata'] =  base64_encode_gzip(metadata_string)
    outputs['userdata'] =  base64_encode_gzip(cps['userdata'])
     
    return outputs

set cloudbase-init guestinfo data

powershell

function handler($context, $inputs) {
  $outputs = @{}
  
  $vc = $inputs.vcenter
  write-host "connecting to vcenter $vc"
  
  connect-viserver -Server $inputs.vcenter -user $inputs.vcenter_username -password $context.getSecret($inputs.vcenter_password) -protocol https -force
  
  $vm = get-vm $inputs.resourceNames[0]
  $vm | new-advancedsetting -name "guestinfo.metadata.encoding" -value "gzip+base64" -confirm:$false
  $vm | new-advancedsetting -name "guestinfo.userdata.encoding" -value "gzip+base64" -confirm:$false
  $vm | new-advancedsetting -name "guestinfo.metadata" -value $inputs.metadata -confirm:$false
  $vm | new-advancedsetting -name "guestinfo.userdata" -value $inputs.userdata -confirm:$false
  
  disconnect-viserver
  
  return $outputs
}

set cloudbase-init guestinfo

flow

---
version: 1
flow:
  flow_start:
    next: action1
  action1:
    action: make_cloudinit_guestinfo_data
    next: action2
  action2:
    action: set_cloudinit_guestinfo_data
    next: flow_end

Subscriptions

create ad computer object

• event topic: Compute Provision
• Workflow
  • Create AD Computer Object

set cloudbase-init data

• event topic: Compute Initial power on
• action
  • set cloudbase-init guestinfo

delete ad computer object

• event topic: Compute removal
• workflow
  • Delete AD Computer Object

[antonioaraujo]

Dear @pizzle85,

I hope you are fine.

First of all, Thank you!. Thank you very much for sharing this detailed guide on Using Aria Automation to Deploy Windows Server using Cloudbase-Init.

I have been fighting with a Windows Server 2022 template and a cloud template for some time to configure hostname, ip address, netmask, default gateway, dns servers, domain, and attach additional disks with its respective label, drive letter, size and storage type. Each disk needs to be initialized and formatted.

I think I have not understood well how Sysprep and Cloudbase-init work together and the moment in which Cloudbase-init commands are executed.

The approach that I am following is using a Windows Server 2022 with Cloudbase-init version 1.6 configured to use OvfService to read metadata. In cloud template's cloudConfig section I am using Multipart as shown below:

formatVersion: 1
inputs:
  description:
    type: string
    title: Description
    description: Deployment description
  cpuCount:
    type: integer
    title: CPU count
    description: CPU count
    minimum: 1
    #maximum: 16 # maximum value defined by the resources
    default: 2
  memory:
    type: integer
    title: Memory in MB
    description: memory in MB
    default: 2048
    minimum: 1024
    #maximum: 16384 # maximum value defined by the resources
  username:
    type: string
    default: winadmin
    title: Username
    description: Username
    readOnly: true
  password:
    type: string
    encrypted: true
    title: Password
    description: Password should have between 16 and 25 characters including at least one  lowercase letter, one uppercase letter, one numeric digit, and one special character.
    default: null
    pattern: ^(?=.*\d)(?=.*[a-z])(?=.*[A-Z])(?=.*[^a-zA-Z0-9])(?!.*\s).{16,25}$
    populateRequiredOnNonDefaultProperties: true
  disks:
    type: array
    title: Enter disks
    description: Enter disks (optional and maximum 5)
    minItems: 0
    maxItems: 5
    items:
      type: object
      properties:
        name:
          type: string
          title: Label
          maxLength: 32
          minLength: 1
          description: Label for the attached disk (string with length between 1 and 32, no special characters). Example Data
          populateRequiredForNestedProperties: true
          pattern: ^[A-Za-z0-9 _.-]{1,32}$
        size:
          type: integer
          title: Size in GB
          default: 10
          minimum: 10
          maximum: 10240
          description: Size in GB
        storage:
          type: string
          title: Storage type
          populateRequiredForNestedProperties: true
          oneOf:
            - title: Silver
              const: Silver
            - title: Gold
              const: Gold
  network:
    type: string
    title: Network
  ipAddress:
    type: string
    title: IP address
    description: Enter IP address
    pattern: ^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$
    populateRequiredOnNonDefaultProperties: true
  netmask:
    type: integer
    description: Enter netmask
    minimum: 0
    maximum: 32
    title: Netmask
    default: 24
  gateway:
    type: string
    title: Default gateway
    description: Enter default gateway
    pattern: ^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$
    populateRequiredOnNonDefaultProperties: true
  dnsServer:
    type: string
    title: DNS servers
    description: DNS servers list (use comma and one space between servers)
    pattern: ^((?:(?:25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)\.){3}(?:25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)|(?:[a-zA-Z0-9-]+\.)+[a-zA-Z]{2,})(, (?:(?:(?:25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)\.){3}(?:25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)|(?:[a-zA-Z0-9-]+\.)+[a-zA-Z]{2,}))*$
  domain:
    type: string
    title: Enter domain
    description: Domain
    default: abc.xyz
    pattern: ^(([a-zA-Z]|[a-zA-Z][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z]|[A-Za-z][A-Za-z0-9\-]*[A-Za-z0-9])$
    populateRequiredOnNonDefaultProperties: true
resources:
  Cloud_vSphere_Machine_1:
    type: Cloud.vSphere.Machine
    properties:
      image: Windows-Server-2022-Cloudbase-init-test_vRA
      cpuCount: ${input.cpuCount}
      totalMemoryMB: ${input.memory}
      attachedDisks: ${map_to_object(resource.disk[*].id, 'source')}
      folderName: ${env.projectName}
      description: ${input.description}
      remoteAccess:
        authentication: usernamePassword
        username: ${input.username}
        password: ${input.password}
      cloudConfig: |
        Content-Type: multipart/mixed; boundary="==NewPart=="
        MIME-Version: 1.0

        --==NewPart==
        Content-Type: text/cloud-config; charset="us-ascii"
        MIME-Version: 1.0
        Content-Transfer-Encoding: 7bit
        Content-Disposition: attachment; filename="cloud-config"

        users:
          - 
            name: ${input.username}
            passwd: ${input.password}

        set_hostname: ${self.resourceName}

        # Write labelNames.json to C:\ on the VM with disk names
        write_files:
          - path: C:\Program Files\Cloudbase Solutions\Cloudbase-Init\LocalScripts\labelNames.json
            content: |
              {
                "volumes": ${to_json(map_to_object(resource.disk[*].name, "name"))}
              }      

        --==NewPart==
        Content-Type: text/x-shellscript; charset="us-ascii"
        MIME-Version: 1.0
        Content-Transfer-Encoding: 7bit
        Content-Disposition: attachment; filename="configuration.ps1"

        #ps1_sysnative
        
        # Remove any trace of password from log file
        (Get-Content "C:\Program Files\Cloudbase Solutions\Cloudbase-Init\log\cloudbase-init.log") -replace "${input.password}","<LOCAL-USER-PASSWORD>" | Set-Content "C:\Program Files\Cloudbase Solutions\Cloudbase-Init\log\cloudbase-init.log" -Verbose

        # Check if disk operations have already been performed
        $diskFlagFile = "C:\Program Files\Cloudbase Solutions\Cloudbase-Init\LocalScripts\diskConfigured.txt"
        if (Test-Path $diskFlagFile) {
          Write-Host "Disk configuration already completed. Skipping disk tasks."
        } else {
          # Read network configuration and apply it
          $adapter = Get-NetAdapter | Where-Object { $_.Name -eq "Ethernet1" }
          $adapter | Remove-NetIpAddress -Confirm:$false
          $adapter | Remove-NetRoute -Confirm:$false
          $adapter | New-NetIpAddress -IPAddress ${input.ipAddress} -PrefixLength ${input.netmask} -DefaultGateway ${input.gateway}
          $adapter | Set-DnsClientServerAddress -ServerAddresses("${input.dnsServer}")
          $adapter | Disable-NetAdapterBinding -ComponentID ms_tcpip6

          # Delay the disk operations to ensure the system has fully booted
          Start-Sleep -Seconds 60

          # Read disk labels from JSON file
          $jsonFile = "C:\Program Files\Cloudbase Solutions\Cloudbase-Init\LocalScripts\labelNames.json"
          if (Test-Path $jsonFile) {
            $jsonContent = Get-Content -Path $jsonFile -Raw | ConvertFrom-Json
            # Extract only the label strings (e.g., "DataSilver", "DataGold")
            $volumes = $jsonContent.volumes | ForEach-Object { $_.name }
          } else {
            Write-Host "Label file not found. Using default labels."
            $volumes = @("Data", "Logs")  # Default labels if the file is not found
          }

          # Initialize and configure the disks
          $drives = Get-Disk | Where-Object { $_.PartitionStyle -eq 'RAW' -and $_.Number -gt 0 }
          $letters = 68  # ASCII 'D'

          for ($i = 0; $i -lt $drives.Count; $i++) {
            $disk = $drives[$i]
            $volumeName = $volumes[$i]

            Initialize-Disk -Number $disk.Number -PartitionStyle GPT -PassThru | `
                New-Partition -UseMaximumSize -AssignDriveLetter | `
                Format-Volume -FileSystem NTFS -NewFileSystemLabel $volumeName -Confirm:$false

            # Assign drive letter (starting from 'D')
            $letter = [char]$letters
            $partition = Get-Partition -DiskNumber $disk.Number | Where-Object DriveLetter -NE $null
            if ($partition) {
                Set-Partition -DriveLetter $partition.DriveLetter -NewDriveLetter $letter
            }

            # Increment drive letter
            $letters++
          }

          # Mark disk tasks as completed by creating a flag file
          New-Item -Path $diskFlagFile -ItemType File -Force
          Write-Host "Disk configuration completed successfully."
        }

        # Eject any removable drives
        $drives = Get-WmiObject Win32_Volume -Filter "DriveType=5"
        $drives | ForEach-Object { (New-Object -ComObject Shell.Application).Namespace(17).ParseName($_.Name).InvokeVerb("Eject") } -ErrorAction SilentlyContinue
      networks:
        - name: ${resource.Cloud_vSphere_Network_1
          assignment: static
          network: ${resource.Cloud_vSphere_Network_1.id}
      storage:
  #      bootDiskCapacityInGB: ${input.bootDiskCapacity}
        constraints:
          - tag: storage-tier:silver
  disk: #type: Cloud.Volume
    type: Cloud.vSphere.Disk
    allocatePerInstance: true
    properties:
      capacityGb: ${input.disks[count.index].size}
      name: ${input.disks[count.index].name}
      count: ${length(input.disks)}
      provisioningType: thin
      SCSIController: SCSI_Controller_0
      unitNumber: ${(count.index+1)}
      constraints:
        - tag: '${input.disks[count.index].storage == "Silver" ? "storage-tier:silver" : (input.disks[count.index].storage == "Gold" ? "storage-tier:gold" :  "storage-tier:silver") }'
  Cloud_vSphere_Network_1:
    type: Cloud.vSphere.Network
    properties:
      networkType: existing
      constraints:
        - tag: ${input.network}

The problem I have is, if I deploy a VM with only one attached disk, then the PowerShell script  described as "configuration-vRA.ps1" do not find any disk (Get-Disk cmdlet), however if I attach three disks they are found and initialize.

I do not know if that is related to a specific stage of the VM boot or it is related to Sysprep configuration.

I would appreciate any comments on this. I mean is there any order in which the VM is booted or Cloudbase-init PowerShell script is executed?

Best regards

Antonio

[qc4vmware]

We are also late to the party in a way... late so far as trying to use cloudbase-init.  We discussed modifying our process and tried briefly but ended up falling back to our existing process we used with vRA 7.  I'm hoping in the upcoming months to try again and this will be a great guide to reference.  Thanks for sharing!  Without the communities I feel like vRA would be impossible to use at times.  I'm thankful for those that are a lot smarter and/or tenacious than me for answering questions and openly sharing anything others think will be helpful.

[Brian MacInnes]

Hi. This article is great.   

I want you to know there is a typo in one of the configs

metadata_services= cloudbaseinit.metadata.services.vmwareguestintoservice.VMwareGuestInfoService

vmwareguestintoservice is supposed to be vmwareguestinfoservice

fixed:   metadata_services=cloudbaseinit.metadata.services.vmwareguestinfoservice.VMwareGuestInfoService

-------------------------------------------