Holodeck Toolkit via VCF Lab Constructor

 View Only
Expand all | Collapse all

Unable to deploy additional Workload Domain

  • 1.  Unable to deploy additional Workload Domain

    Posted 24 days ago

    Hello,

    First of all, thanks a lot for your work you put into the evolution of the holodeck, working with it since the first version.
    At the weekend, I gave VCF 9.0 Holodeck a go, and everything went fine.

    I deployed with Parameter -ManagementOnly to minimize the impact on my physical host's resources.
    After deployment of my Management Domain went well, I would like to deploy a additional Workload Domain.

    For that, I used the function "Deploy-ManagementDomain -Version 9.0"
    Sadly, this went to an error, and I have no clue why.
    So maybe someone of you could help me and guide me in the right direction?

    The following error I got:

    PS /root> Deploy-ManagementDomain -Version 9.0 -Verbose                             
     
    cmdlet Deploy-ManagementDomain at command pipeline position 1
    Supply values for the following parameters:
    site: A
    14-07-2025 05:59:51  SddcMgmtDomain[4247]: [INFO] Beginning SDDC Management Domain Deployment
    14-07-2025 05:59:51  SddcMgmtDomain[4247]: [INFO] Verifying VCF Installer availability
    VERBOSE: Requested HTTP/1.1 POST with 69-byte payload
    VERBOSE: Received HTTP/1.1 response of content type application/json of unknown size
    VERBOSE: Content encoding: utf-8
    14-07-2025 05:59:51  SddcMgmtDomain[4247]: [INFO] VCF Installer is now ready
    14-07-2025 05:59:51  SddcMgmtDomain[4247]: [INFO] Install VCF bundles in VCF Installer
    14-07-2025 05:59:51  SddcMgmtDomain[4247]: [INFO] Getting Bundle Details
    VERBOSE: Requested HTTP/1.1 GET with 0-byte payload
    VERBOSE: Received HTTP/1.1 response of content type application/json of unknown size
    VERBOSE: Content encoding: utf-8
    14-07-2025 05:59:51  SddcMgmtDomain[4247]: [INFO] Received Bundles. Checking if all VCF 9 bundles are available
    14-07-2025 05:59:51  SddcMgmtDomain[4247]: [INFO] Received all Bundle Details
    14-07-2025 05:59:51  SddcMgmtDomain[4247]: [INFO] Initiating bundle download
    VERBOSE: Requested HTTP/1.1 PATCH with 57-byte payload
    VERBOSE: Received HTTP/1.1 response of content type application/json of unknown size
    14-07-2025 05:59:51  SddcMgmtDomain[4247]: [ERROR] Error initiating download for component:                             
    14-07-2025 05:59:51  SddcMgmtDomain[4247]: [ERROR] Encountered Error: 
    Exception: Exiting

    My Offline Depot, which I used for Deployment, is still available for sure.



  • 2.  RE: Unable to deploy additional Workload Domain

    Broadcom Employee
    Posted 24 days ago
    Edited by Dhruv Tyagi 24 days ago

    Hi Manuel,

    Thank you for being an active community member and working with Holodeck through all the years of its evolution. 

    DeployManagementDomain function as the name suggests deploys the management domain. For deploying the workload domain manually, you can use the command Deploy-WorkloadDomain -Version 9.0 -Site a

    As you may already be aware, if you wish to deploy the full stack, you can run the New-HolodeckInstance command without the -ManagementOnly parameter to deploy both management and workload domain.

    Please let us know if you have any queries.




  • 3.  RE: Unable to deploy additional Workload Domain

    Posted 23 days ago

    Hi Dhruv,

    Thanks a lot for your fast response. For sure, I'm aware of the possibility that a New-Holodeck Instance can be deployed without the -ManagementOnly parameter.

    But I want to be modular here, sometimes you need an instance with WLD and sometimes without.
    So in my case, I would like to be able to deploy the WLD afterwards, and I thought with the command/function I mentioned it would be possible.

    Is there a way to figure out with Offline-Depod is addressed in my case? Is there a variable or config file?




  • 4.  RE: Unable to deploy additional Workload Domain

    Broadcom Employee
    Posted 23 days ago
    Edited by Dhruv Tyagi 23 days ago

    Hi Manuel,

    Understood. Yes, you can use the Deploy-WorkloadDomain command to deploy the workload domain afterwards.

    The offline depot is used for management domain deployment by VCF Installer and not needed for workload domain, so you should be good to proceed since your management domain is already deployed. You can later use the offline depot to store VCF Ops for Networks, VCF Ops for Logs, HCX amongst other components that can be deployed manually (not deployed as part of the workload domain automatically).

    As for checks for offline depot, we will introduce more guardrails and checks in Holodeck in upcoming releases/patches to ensure the user has set up the offline depot properly and has the required binaries in the right place.




  • 5.  RE: Unable to deploy additional Workload Domain

    Posted 23 days ago

    Ok, so I'm a little bit more confused - why am I getting the following error message:

    PS /root> Deploy-ManagementDomain -Version 9.0
     
    cmdlet Deploy-ManagementDomain at command pipeline position 1
    Supply values for the following parameters:
    site: a
    14-07-2025 08:15:01  SddcMgmtDomain[4247]: [INFO] Beginning SDDC Management Domain Deployment
    14-07-2025 08:15:01  SddcMgmtDomain[4247]: [INFO] Verifying VCF Installer availability
    14-07-2025 08:15:01  SddcMgmtDomain[4247]: [INFO] VCF Installer is now ready
    14-07-2025 08:15:01  SddcMgmtDomain[4247]: [INFO] Install VCF bundles in VCF Installer
    14-07-2025 08:15:01  SddcMgmtDomain[4247]: [INFO] Getting Bundle Details
    14-07-2025 08:15:02  SddcMgmtDomain[4247]: [INFO] Received Bundles. Checking if all VCF 9 bundles are available
    14-07-2025 08:15:02  SddcMgmtDomain[4247]: [INFO] Received all Bundle Details
    14-07-2025 08:15:02  SddcMgmtDomain[4247]: [INFO] Initiating bundle download
    14-07-2025 08:15:02  SddcMgmtDomain[4247]: [ERROR] Error initiating download for component:                             
    14-07-2025 08:15:02  SddcMgmtDomain[4247]: [ERROR] Encountered Error: 
    Exception: Exiting




  • 6.  RE: Unable to deploy additional Workload Domain

    Broadcom Employee
    Posted 23 days ago

    Hi Manuel,

    You mentioned:

    I deployed with Parameter -ManagementOnly to minimize the impact on my physical host's resources.
    After deployment of my Management Domain went well, I would like to deploy a additional Workload Domain.

    Why are you running Deploy-ManagementDomain again? If you want to deploy the workload domain, you need to run Deploy-WorkloadDomain.




  • 7.  RE: Unable to deploy additional Workload Domain

    Posted 23 days ago

    The answer is easy, why not do so - time.
    Deployment of the Management Domain took around 10 Hours with Automation.
    In addition, you have to do some additional stuff within Automation to prepare the environment be ready for a customer Demo who wants to see SDDC in full action.
    So redeployment of the management stack is not an option for me right now as I prepare for a customer demo on Thursday.




  • 8.  RE: Unable to deploy additional Workload Domain

    Broadcom Employee
    Posted 23 days ago

    Hi Manuel, 

    I think there is some confusion. What you have already run:

    New-HoloDeckInstance -Version 9.0 -ManagementOnly


    Current state of your environment: VCF 9.0 Management domain is deployed successfully.

    Your ask now: I have already deployed VCF 9.0 management domain using Holodeck. Now I want to deploy a workload domain to the same environment.

    The answer: Run Deploy-WorkloadDomain command to deploy an additional workload domain to your existing setup with management domain deployed.

    This is assuming you still have the same powershell session open where you ran New-HolodeckInstance command. If you don't, then you can open a new powershell session, run Get-HoloDeckConfig, then Import-HoloDeckConfig -ConfigID <your-ConfigID>, then run Deploy-WorkloadDomain command.

    Please let me know if there is still any confusion




  • 9.  RE: Unable to deploy additional Workload Domain

    Posted 23 days ago

    Hi Dhruv.

    You summarized absolutely correctly. I still have the same PowerShell session open, and HoloDeckConfig is loaded.
    Sadly, I got the error Message I've posted before.

    Here to summarize:

    S /root> Get-HoloDeckConfig                    
     
    ConfigID    : nqxi
    Description : MB-Labs-03
    ConfigPath  : /holodeck-runtime/config/nqxi.json
    Instance    : holo
    Created     : 7/13/2025 3:55:19 PM
     
    PS /root> $config
     
    Title               : HoloDeck 9.0 Config File
    Name                : 
    ConfigId            : nqxi
    InstanceId          : holo
    description         : MB-Labs-03
    ConfigPath          : /holodeck-runtime/config/nqxi.json
    masterPassword      : VMware123!VMware123!
    vSANMode            : ESA
    rootFolderPath      : /holodeck-runtime
    rootFolderName      : holodeck-runtime
    output              : output
    logs                : /holodeck-runtime/logs/holodeck-runtime-log_13Jul2025_08_32.log
    state               : /holodeck-runtime/state/holodeck-runtime-state_13Jul2025_08_32_42.json
    bin                 : bin
    templates           : templates
    holorouterStatus    : Configured
    isPreCheckCompleted : true
    Target              : @{hostname=lab-esx07.mb.lab; username=root; password=XXXX; targetHostApiType=HostAgent; 
                          datacenter=ha-datacenter; cluster=; datastore=Lab-ESX07-RAID5; networkPortGroup=VLC-A-PG; 
                          isDeprecatedCPU=True}
    holodeck-sddc       : @{Site-A=; Site-B=}
     
    PS /root> Deploy-ManagementDomain -Version 9.0
     
    cmdlet Deploy-ManagementDomain at command pipeline position 1
    Supply values for the following parameters:
    site: a
    14-07-2025 15:59:49  SddcMgmtDomain[4247]: [INFO] Beginning SDDC Management Domain Deployment
    14-07-2025 15:59:49  SddcMgmtDomain[4247]: [INFO] Verifying VCF Installer availability
    14-07-2025 15:59:49  SddcMgmtDomain[4247]: [INFO] VCF Installer is now ready
    14-07-2025 15:59:49  SddcMgmtDomain[4247]: [INFO] Install VCF bundles in VCF Installer
    14-07-2025 15:59:49  SddcMgmtDomain[4247]: [INFO] Getting Bundle Details
    14-07-2025 15:59:49  SddcMgmtDomain[4247]: [INFO] Received Bundles. Checking if all VCF 9 bundles are available
    14-07-2025 15:59:49  SddcMgmtDomain[4247]: [INFO] Received all Bundle Details
    14-07-2025 15:59:49  SddcMgmtDomain[4247]: [INFO] Initiating bundle download
    14-07-2025 15:59:49  SddcMgmtDomain[4247]: [ERROR] Error initiating download for component:                             
    14-07-2025 15:59:49  SddcMgmtDomain[4247]: [ERROR] Encountered Error: 
    Exception: Exiting
    PS /root> 



  • 10.  RE: Unable to deploy additional Workload Domain

    Broadcom Employee
    Posted 23 days ago

    Hi Manuel,

    You need to run Deploy-WorkloadDomain command




  • 11.  RE: Unable to deploy additional Workload Domain

    Posted 22 days ago

    Hi Dhruv,

    Thanks a lot for highlighting. What a bad fault :(
    I tried the command you mentioned, sadly, I got another error.

    PS /root> $config
     
    Title               : HoloDeck 9.0 Config File
    Name                : 
    ConfigId            : nqxi
    InstanceId          : holo
    description         : MB-Labs-03
    ConfigPath          : /holodeck-runtime/config/nqxi.json
    masterPassword      : VMware123!VMware123!
    vSANMode            : ESA
    rootFolderPath      : /holodeck-runtime
    rootFolderName      : holodeck-runtime
    output              : output
    logs                : /holodeck-runtime/logs/holodeck-runtime-log_13Jul2025_08_32.log
    state               : /holodeck-runtime/state/holodeck-runtime-state_13Jul2025_08_32_42.json
    bin                 : bin
    templates           : templates
    holorouterStatus    : Configured
    isPreCheckCompleted : true
    Target              : @{hostname=lab-esx07.mb.lab; username=root; password=XXXXX; targetHostApiType=HostAgent; 
                          datacenter=ha-datacenter; cluster=; datastore=Lab-ESX07-RAID5; networkPortGroup=VLC-A-PG; 
                          isDeprecatedCPU=True}
    holodeck-sddc       : @{Site-A=; Site-B=}
     
    PS /root> Deploy-WorkloadDomain -Version 9.0  
     
    cmdlet Deploy-WorkloadDomain at command pipeline position 1
    Supply values for the following parameters:
    site: a
    15-07-2025 08:29:25  SddcWkldDomain[4247]: [INFO] Beginning SDDC Workload Domain Deployment
    15-07-2025 08:29:25  SddcWkldDomain[4247]: [INFO] Getting SDDC Manager API Token
    15-07-2025 08:29:25  SddcWkldDomain[4247]: [INFO] Received SDDC Manager API Token
    15-07-2025 08:29:25  SddcWkldDomain[4247]: [INFO] Beginning host commissioning to SDDC Manager
    15-07-2025 08:29:25  SddcWkldDomain[4247]: [INFO] Get Network Pool ID
    15-07-2025 08:29:26  SddcWkldDomain[4247]: [INFO] Received network pool ID successfully
    15-07-2025 08:29:26  SddcWkldDomain[4247]: [INFO] Commissioning hosts to SDDC Manager for workload domain
    15-07-2025 08:29:40  SddcWkldDomain[4247]: [ERROR] Error:                                                               
    {
      "errorCode": "REST_INVALID_API_INPUT",
      "arguments": [],
      "message": "Invalid input",
      "remediationMessage": "Enter correct API input",
      "nestedErrors": [
        {
          "errorCode": "ANNOTATIONS_MISMATCH",
          "arguments": [
            "hosts.HOST_FQDN_BLANK,hosts.HOST_FQDN_BLANK,hosts.HOST_FQDN_BLANK"
          ],
          "message": "Following conditions do not match - hosts.HOST_FQDN_BLANK,hosts.HOST_FQDN_BLANK,hosts.HOST_FQDN_BLANK"
        }
      ],
      "referenceToken": "8O68I0"
    }
    15-07-2025 08:29:40  SddcWkldDomain[4247]: [ERROR] Error Message: 
    {
      "errorCode": "REST_INVALID_API_INPUT",
      "arguments": [],
      "message": "Invalid input",
      "remediationMessage": "Enter correct API input",
      "nestedErrors": [
        {
          "errorCode": "ANNOTATIONS_MISMATCH",
          "arguments": [
            "hosts.HOST_FQDN_BLANK,hosts.HOST_FQDN_BLANK,hosts.HOST_FQDN_BLANK"
          ],
          "message": "Following conditions do not match - hosts.HOST_FQDN_BLANK,hosts.HOST_FQDN_BLANK,hosts.HOST_FQDN_BLANK"
        }
      ],
      "referenceToken": "8O68I0"
    }
    15-07-2025 08:29:40  SddcWkldDomain[4247]: [ERROR] See the operationsmanager or domainmanager log on SDDC for more information
    15-07-2025 08:29:40  SddcWkldDomain[4247]: [ERROR] SSH or console to sddcmanager-a.site-a.vcf.lab
    15-07-2025 08:29:40  SddcWkldDomain[4247]: [ERROR] Logs are located at /var/log/vmware/vcf
    Exception: Exiting
    PS /root> 






  • 12.  RE: Unable to deploy additional Workload Domain

    Broadcom Employee
    Posted 22 days ago

    Hi Manuel,

    Understood. Since you ran New-HoloDeckInstance -ManagementOnly, 3 ESX hosts for workload domain were not created and hence you are seeing this error. I would suggest you cleanup and deploy full stack again as the process for just setting up workload domain can be a little complicated to do manually.




  • 13.  RE: Unable to deploy additional Workload Domain

    Posted 22 days ago

    Hey Dhruv, not the answer I was hoping for. As I mentioned before, a limited time frame is given and I did some customization within automation. To redeploy the management domain is not a suitable way for me. Is there a way to add the three hosts for WLD and then deploy the WLD? I think holodeck should provide this flexibility. When not given in the actual version, this may be a suggestion for an upcoming release.




  • 14.  RE: Unable to deploy additional Workload Domain

    Broadcom Employee
    Posted 22 days ago
    Edited by Dhruv Tyagi 22 days ago

    If you really want to go this route, here's everything you need to do.

    Assuming the config is already loaded in your powershell session.

    1. New-HoloDeckESXiNodes -VIDomain Workload -Site a

    This will deploy 3 ESXi VMs without an ISO.

    2. Open the file /holodeck-runtime/output/output.txt and ensure only esx 5,6 and 7 are listed in there. Remove any other entries and save the file

    3. Run New-HoloDeckESXiISO -Site a

    This adds the ISO to the 3 hosts for workload domain. You need to be careful and ensure no other hosts are in the output.txt otherwise they will be booted with the new ISO and you will loose any appliances on those ESX and won't be able to recover.

    Now you have the ESX hosts for workload domain ready. But your config was not set up for full stack. So you need to update the config.

    4. Run the following:

    $site = "a"
    
    $siteCode = "Site-$site"
    
    $version= $config.'holodeck-sddc'.$siteCode.version
    $HoloSubnet = Get-HoloDeckSubnet -Site $site
    $holoDeckAppInfo = Get-HoloDeckAppNetwork -Site $site
    $HoloOverlay = Get-HoloDeckOverlaySubnet -Site $site
    
    
    $config.'holodeck-sddc'.$siteCode.isManagementOnly = "false"
    $config | Convertto-Json -Depth 100 |Set-Content $Config.ConfigPath 
    $Global:config = Get-Content $Config.configPath |ConvertFrom-Json 
    5. If you want the workload domain to be deployed with NSX Edge cluster, run the following:
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain' | % {$_.asn = (Get-HoloDeckBGPConfig -Site $site).Edge_Wld_AS}
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0] | % {$_.edgeNodeName = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-wld01-01${site}"}).FQDN }
    
    
    if($version -eq "9.0"){
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain' | % {$_.tier0ServicesHighAvailability = "ACTIVE_STANDBY"}
    }
    
    
    $edgeTepSubnet = (($HoloSubnet | Where-Object {$_.Name -eq "etep-${site}"}).Subnet).Split('/')[1]
    $edgeNode1Tep1IP = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-tep01-wld01-01${site}"}).IP
    $edgeNode1Tep2IP = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-tep02-wld01-01${site}"}).IP
    $edgeNode2Tep1IP = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-tep01-wld01-02${site}"}).IP
    $edgeNode2Tep2IP = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-tep02-wld01-02${site}"}).IP
    $edgeNode1Tep1IPwithMask = "${edgeNode1Tep1IP}/${edgeTepSubnet}"
    $edgeNode1Tep2IPwithMask = "${edgeNode1Tep2IP}/${edgeTepSubnet}"
    $edgeNode2Tep1IPwithMask = "${edgeNode2Tep1IP}/${edgeTepSubnet}"
    $edgeNode2Tep2IPwithMask = "${edgeNode2Tep2IP}/${edgeTepSubnet}"
    
    
    $mgmtSubnet = (($HoloSubnet | Where-Object {$_.Name -eq "mgmt-${site}"}).Subnet).Split('/')[1]
    $edgeNode1MgmtIP = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-wld01-01${site}"}).IP
    $edgeNode2MgmtIP = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-wld01-02${site}"}).IP
    $edgeNode1MgmtIPwithMask = "${edgeNode1MgmtIP}/${mgmtSubnet}"
    $edgeNode2MgmtIPwithMask = "${edgeNode2MgmtIP}/${mgmtSubnet}"
    
    
    $edgeUplink1Subnet = (($HoloSubNet | Where-Object {$_.Name -eq "uplink01-${site}"}).Subnet).Split('/')[1]
    $edgeUplink1PeerIP = ($HoloSubNet | Where-Object {$_.Name -eq "uplink01-${site}"}).Gateway
    $edgeUplink2Subnet = (($HoloSubNet | Where-Object {$_.Name -eq "uplink02-${site}"}).Subnet).Split('/')[1]
    $edgeUplink2PeerIP = ($HoloSubNet | Where-Object {$_.Name -eq "uplink02-${site}"}).Gateway
    $edgeUplink1PeerIPwithMask = "${edgeUplink1PeerIP}/${edgeUplink1Subnet}"
    $edgeUplink2PeerIPwithMask = "${edgeUplink2PeerIP}/${edgeUplink1Subnet}"
    
    
    $edgeUplink1InterfaceSubnet = (($HoloSubnet | Where-Object {$_.Name -eq "uplink01-${site}"}).Subnet).Split('/')[1]
    $edgeUplink2InterfaceSubnet = (($HoloSubnet | Where-Object {$_.Name -eq "uplink02-${site}"}).Subnet).Split('/')[1]
    $edgeNode1Uplink1InterfaceIP = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-uplink01-wld01-01${site}"}).IP
    $edgeNode1Uplink2InterfaceIP = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-uplink02-wld01-01${site}"}).IP
    $edgeNode2Uplink1InterfaceIP = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-uplink01-wld01-02${site}"}).IP
    $edgeNode2Uplink2InterfaceIP = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-uplink02-wld01-02${site}"}).IP
    $edgeNode1Uplink1InterfaceIPwithMask = "${edgeNode1Uplink1InterfaceIP}/${edgeUplink1InterfaceSubnet}"
    $edgeNode1Uplink2InterfaceIPwithMask = "${edgeNode1Uplink2InterfaceIP}/${edgeUplink2InterfaceSubnet}"
    $edgeNode2Uplink1InterfaceIPwithMask = "${edgeNode2Uplink1InterfaceIP}/${edgeUplink1InterfaceSubnet}"
    $edgeNode2Uplink2InterfaceIPwithMask = "${edgeNode2Uplink2InterfaceIP}/${edgeUplink2InterfaceSubnet}"
    
    
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0] | % {$_.edgeTep1IP = $edgeNode1Tep1IPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0] | % {$_.edgeTep2IP = $edgeNode1Tep2IPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0] | % {$_.edgeTepGateway = ($HoloSubNet|where-object {$_.Name -eq "etep-${site}"}).gateway}
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0] | % {$_.edgeTepVlan = ($HoloSubNet|where-object {$_.Name -eq "etep-${site}"}).VLAN_ID }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0] | % {$_.managementGateway = ($HoloSubNet | Where-Object {$_.Name -eq "mgmt-${site}"}).Gateway}
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0] | % {$_.managementIP = $edgeNode1MgmtIPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0].uplinkNetwork[0] | % {$_.asnPeer = (Get-HoloDeckBGPConfig -Site $site).Router_AS }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0].uplinkNetwork[0] | % {$_.bgpPeerPassword = (Get-HoloDeckBGPConfig -Site $site).BGP_Password }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0].uplinkNetwork[0] | % {$_.peerIP = $edgeUplink1PeerIPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0].uplinkNetwork[0] | % {$_.uplinkInterfaceIP = $edgeNode1Uplink1InterfaceIPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0].uplinkNetwork[0] | % {$_.uplinkVlan = ($HoloSubNet | Where-Object {$_.Name -eq "uplink01-${site}"}).VLAN_ID }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0].uplinkNetwork[1] | % {$_.asnPeer = (Get-HoloDeckBGPConfig -Site $site).Router_AS }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0].uplinkNetwork[1] | % {$_.bgpPeerPassword = (Get-HoloDeckBGPConfig -Site $site).BGP_Password }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0].uplinkNetwork[1] | % {$_.peerIP = $edgeUplink2PeerIPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0].uplinkNetwork[1] | % {$_.uplinkInterfaceIP = $edgeNode1Uplink2InterfaceIPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0].uplinkNetwork[1] | % {$_.uplinkVlan = ($HoloSubNet | Where-Object {$_.Name -eq "uplink02-${site}"}).VLAN_ID }
    
    
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1] | % {$_.edgeNodeName = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-wld01-02${site}"}).FQDN }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1] | % {$_.edgeTep1IP = $edgeNode2Tep1IPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1] | % {$_.edgeTep2IP = $edgeNode2Tep2IPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1] | % {$_.edgeTepGateway = ($HoloSubNet|where-object {$_.Name -eq "etep-${site}"}).gateway}
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1] | % {$_.edgeTepVlan = ($HoloSubNet|where-object {$_.Name -eq "etep-${site}"}).VLAN_ID }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1] | % {$_.managementGateway = ($HoloSubNet | Where-Object {$_.Name -eq "mgmt-${site}"}).Gateway}
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1] | % {$_.managementIP = $edgeNode2MgmtIPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1].uplinkNetwork[0] | % {$_.asnPeer = (Get-HoloDeckBGPConfig -Site $site).Router_AS }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1].uplinkNetwork[0] | % {$_.bgpPeerPassword = (Get-HoloDeckBGPConfig -Site $site).BGP_Password }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1].uplinkNetwork[0] | % {$_.peerIP = $edgeUplink1PeerIPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1].uplinkNetwork[0] | % {$_.uplinkInterfaceIP = $edgeNode2Uplink1InterfaceIPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1].uplinkNetwork[0] | % {$_.uplinkVlan = ($HoloSubNet | Where-Object {$_.Name -eq "uplink01-${site}"}).VLAN_ID }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1].uplinkNetwork[1] | % {$_.asnPeer = (Get-HoloDeckBGPConfig -Site $site).Router_AS }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1].uplinkNetwork[1] | % {$_.bgpPeerPassword = (Get-HoloDeckBGPConfig -Site $site).BGP_Password }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1].uplinkNetwork[1] | % {$_.peerIP = $edgeUplink2PeerIPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1].uplinkNetwork[1] | % {$_.uplinkInterfaceIP = $edgeNode2Uplink2InterfaceIPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1].uplinkNetwork[1] | % {$_.uplinkVlan = ($HoloSubNet | Where-Object {$_.Name -eq "uplink02-${site}"}).VLAN_ID }
    6. If you want to deploy supervisor in the workload domain, then run the following:
    Write-Log -Message "Setting up Supervisor Spec"
    
    
    $supervisorActivationSpec = @{
    "supervisorName" = "supervisor"
    "serviceCidr" = @{
    "address" = "10.96.0.0"
    "prefix" = 23
    }
    "zoneName" = "z-wld-${site}"
    "managementNetwork" = @{
    "details" = @{
    "vlanId" = ($HoloSubNet|where-object {$_.Name -eq "mgmt-${site}"}).VLAN_ID
    "gateway" = ($HoloSubNet|where-object {$_.Name -eq "mgmt-${site}"}).Gateway
    "netMask" = ($HoloSubNet|where-object {$_.Name -eq "mgmt-${site}"}).SubnetMask
    "vdsName" = "vds01-wld01-01${site}"
    }
    "controlPlaneIpRange" = @{
    "startIpAddress" = ($holoDeckAppInfo |where-object {$_.hostname -eq "supervisor"}).IP
    "endIpAddress" = ($holoDeckAppInfo |where-object {$_.hostname -eq "SUP upgrade IP"}).IP
    }
    }
    "vpcNetwork" = @{
    "privateTransitNetworkCidr" = @{
    "address" = "172.16.100.0"
    "prefix" = 24
    }
    "privateCidr" = @{
    "address" = "172.16.200.0"
    "prefix" = 24
    }
    "dnsServers" = @(($HoloSubNet|where-object {$_.Name -eq "mgmt-${site}"}).DNS_Server)
    "ntpServers" = @(($HoloSubNet|where-object {$_.Name -eq "mgmt-${site}"}).NTP_Server)
    }
    }
    
    
    $config.'holodeck-sddc'.$sitecode.'vcf-installer-workload-domain'.'workload-domain-spec'.'computeSpec'.clusterSpecs[0] | Add-Member -MemberType NoteProperty -Name "supervisorActivationSpec" -Value $supervisorActivationSpec -Force
    7. Finally, for the workload domain config, run the following:
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain' | % {$_.asn = (Get-HoloDeckBGPConfig -Site $site).Edge_Wld_AS}
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0] | % {$_.edgeNodeName = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-wld01-01${site}"}).FQDN }
    
    
    if($version -eq "9.0"){
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain' | % {$_.tier0ServicesHighAvailability = "ACTIVE_STANDBY"}
    }
    
    
    $edgeTepSubnet = (($HoloSubnet | Where-Object {$_.Name -eq "etep-${site}"}).Subnet).Split('/')[1]
    $edgeNode1Tep1IP = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-tep01-wld01-01${site}"}).IP
    $edgeNode1Tep2IP = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-tep02-wld01-01${site}"}).IP
    $edgeNode2Tep1IP = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-tep01-wld01-02${site}"}).IP
    $edgeNode2Tep2IP = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-tep02-wld01-02${site}"}).IP
    $edgeNode1Tep1IPwithMask = "${edgeNode1Tep1IP}/${edgeTepSubnet}"
    $edgeNode1Tep2IPwithMask = "${edgeNode1Tep2IP}/${edgeTepSubnet}"
    $edgeNode2Tep1IPwithMask = "${edgeNode2Tep1IP}/${edgeTepSubnet}"
    $edgeNode2Tep2IPwithMask = "${edgeNode2Tep2IP}/${edgeTepSubnet}"
    
    
    $mgmtSubnet = (($HoloSubnet | Where-Object {$_.Name -eq "mgmt-${site}"}).Subnet).Split('/')[1]
    $edgeNode1MgmtIP = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-wld01-01${site}"}).IP
    $edgeNode2MgmtIP = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-wld01-02${site}"}).IP
    $edgeNode1MgmtIPwithMask = "${edgeNode1MgmtIP}/${mgmtSubnet}"
    $edgeNode2MgmtIPwithMask = "${edgeNode2MgmtIP}/${mgmtSubnet}"
    
    
    $edgeUplink1Subnet = (($HoloSubNet | Where-Object {$_.Name -eq "uplink01-${site}"}).Subnet).Split('/')[1]
    $edgeUplink1PeerIP = ($HoloSubNet | Where-Object {$_.Name -eq "uplink01-${site}"}).Gateway
    $edgeUplink2Subnet = (($HoloSubNet | Where-Object {$_.Name -eq "uplink02-${site}"}).Subnet).Split('/')[1]
    $edgeUplink2PeerIP = ($HoloSubNet | Where-Object {$_.Name -eq "uplink02-${site}"}).Gateway
    $edgeUplink1PeerIPwithMask = "${edgeUplink1PeerIP}/${edgeUplink1Subnet}"
    $edgeUplink2PeerIPwithMask = "${edgeUplink2PeerIP}/${edgeUplink1Subnet}"
    
    
    $edgeUplink1InterfaceSubnet = (($HoloSubnet | Where-Object {$_.Name -eq "uplink01-${site}"}).Subnet).Split('/')[1]
    $edgeUplink2InterfaceSubnet = (($HoloSubnet | Where-Object {$_.Name -eq "uplink02-${site}"}).Subnet).Split('/')[1]
    $edgeNode1Uplink1InterfaceIP = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-uplink01-wld01-01${site}"}).IP
    $edgeNode1Uplink2InterfaceIP = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-uplink02-wld01-01${site}"}).IP
    $edgeNode2Uplink1InterfaceIP = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-uplink01-wld01-02${site}"}).IP
    $edgeNode2Uplink2InterfaceIP = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-uplink02-wld01-02${site}"}).IP
    $edgeNode1Uplink1InterfaceIPwithMask = "${edgeNode1Uplink1InterfaceIP}/${edgeUplink1InterfaceSubnet}"
    $edgeNode1Uplink2InterfaceIPwithMask = "${edgeNode1Uplink2InterfaceIP}/${edgeUplink2InterfaceSubnet}"
    $edgeNode2Uplink1InterfaceIPwithMask = "${edgeNode2Uplink1InterfaceIP}/${edgeUplink1InterfaceSubnet}"
    $edgeNode2Uplink2InterfaceIPwithMask = "${edgeNode2Uplink2InterfaceIP}/${edgeUplink2InterfaceSubnet}"
    
    
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0] | % {$_.edgeTep1IP = $edgeNode1Tep1IPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0] | % {$_.edgeTep2IP = $edgeNode1Tep2IPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0] | % {$_.edgeTepGateway = ($HoloSubNet|where-object {$_.Name -eq "etep-${site}"}).gateway}
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0] | % {$_.edgeTepVlan = ($HoloSubNet|where-object {$_.Name -eq "etep-${site}"}).VLAN_ID }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0] | % {$_.managementGateway = ($HoloSubNet | Where-Object {$_.Name -eq "mgmt-${site}"}).Gateway}
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0] | % {$_.managementIP = $edgeNode1MgmtIPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0].uplinkNetwork[0] | % {$_.asnPeer = (Get-HoloDeckBGPConfig -Site $site).Router_AS }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0].uplinkNetwork[0] | % {$_.bgpPeerPassword = (Get-HoloDeckBGPConfig -Site $site).BGP_Password }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0].uplinkNetwork[0] | % {$_.peerIP = $edgeUplink1PeerIPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0].uplinkNetwork[0] | % {$_.uplinkInterfaceIP = $edgeNode1Uplink1InterfaceIPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0].uplinkNetwork[0] | % {$_.uplinkVlan = ($HoloSubNet | Where-Object {$_.Name -eq "uplink01-${site}"}).VLAN_ID }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0].uplinkNetwork[1] | % {$_.asnPeer = (Get-HoloDeckBGPConfig -Site $site).Router_AS }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0].uplinkNetwork[1] | % {$_.bgpPeerPassword = (Get-HoloDeckBGPConfig -Site $site).BGP_Password }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0].uplinkNetwork[1] | % {$_.peerIP = $edgeUplink2PeerIPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0].uplinkNetwork[1] | % {$_.uplinkInterfaceIP = $edgeNode1Uplink2InterfaceIPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[0].uplinkNetwork[1] | % {$_.uplinkVlan = ($HoloSubNet | Where-Object {$_.Name -eq "uplink02-${site}"}).VLAN_ID }
    
    
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1] | % {$_.edgeNodeName = ($holoDeckAppInfo | Where-Object {$_.Hostname -eq "edge-wld01-02${site}"}).FQDN }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1] | % {$_.edgeTep1IP = $edgeNode2Tep1IPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1] | % {$_.edgeTep2IP = $edgeNode2Tep2IPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1] | % {$_.edgeTepGateway = ($HoloSubNet|where-object {$_.Name -eq "etep-${site}"}).gateway}
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1] | % {$_.edgeTepVlan = ($HoloSubNet|where-object {$_.Name -eq "etep-${site}"}).VLAN_ID }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1] | % {$_.managementGateway = ($HoloSubNet | Where-Object {$_.Name -eq "mgmt-${site}"}).Gateway}
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1] | % {$_.managementIP = $edgeNode2MgmtIPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1].uplinkNetwork[0] | % {$_.asnPeer = (Get-HoloDeckBGPConfig -Site $site).Router_AS }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1].uplinkNetwork[0] | % {$_.bgpPeerPassword = (Get-HoloDeckBGPConfig -Site $site).BGP_Password }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1].uplinkNetwork[0] | % {$_.peerIP = $edgeUplink1PeerIPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1].uplinkNetwork[0] | % {$_.uplinkInterfaceIP = $edgeNode2Uplink1InterfaceIPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1].uplinkNetwork[0] | % {$_.uplinkVlan = ($HoloSubNet | Where-Object {$_.Name -eq "uplink01-${site}"}).VLAN_ID }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1].uplinkNetwork[1] | % {$_.asnPeer = (Get-HoloDeckBGPConfig -Site $site).Router_AS }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1].uplinkNetwork[1] | % {$_.bgpPeerPassword = (Get-HoloDeckBGPConfig -Site $site).BGP_Password }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1].uplinkNetwork[1] | % {$_.peerIP = $edgeUplink2PeerIPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1].uplinkNetwork[1] | % {$_.uplinkInterfaceIP = $edgeNode2Uplink2InterfaceIPwithMask }
    $config.'holodeck-sddc'.$sitecode.'nsx-edge-cluster-wkld-domain'.edgeNodeSpecs[1].uplinkNetwork[1] | % {$_.uplinkVlan = ($HoloSubNet | Where-Object {$_.Name -eq "uplink02-${site}"}).VLAN_ID }
    
    
    $config | Convertto-Json -Depth 100 |Set-Content $Config.ConfigPath #updating config file
    $Global:config = Get-Content $Config.configPath |ConvertFrom-Json #creating new global variable
    Now your config is set up.
    8. Run Deploy-WorkloadDomain -Version 9.0 -Site a




  • 15.  RE: Unable to deploy additional Workload Domain

    Posted 20 days ago

    Hi Dhruv,

    Thank you a lot for this detailed answer.
    Is there somewhere a reference about these additional commands?

    best
    Manuel





  • 16.  RE: Unable to deploy additional Workload Domain

    Broadcom Employee
    Posted 20 days ago

    Hi Manuel,

    No, these are not meant to be used directly by customers. We will build more commands as we go along to cater to these sorts of requirements