View Only
  • 1.  Incomplete vCenter Update - Cant finalize it or revert

    Posted Sep 24, 2021 06:54 PM

    Hi guys

    I wanted to udate a vCenter Appliance 6.7 Update 3n (  to vCenter Appliance 6.7 Update 3o ( Unfortunately, something went wrong and the update got stuck and never finished. So once I rebooted the appliance, it came up normally and also the vCenter services came up and I can access the vCenter.

    Within the vSphere Client it show the build of 6.7 U3o, but on the VM console it still shows U3n. The problem is, I cant revert to snapshot (shame on me) and I cant restore from backup becuase the last one is from July as the customer disconnected the NFS store where the beackups were made. Perfect, I know ...


    I tried re-running the update, but it fails:


    . I tried

    software-packages install --iso


    software-packages stage --iso followed by software-packages install --staged

    Both leads to the same output. This is what can be found in /var/log/vmware/applmgmt/software-packages.log:


    I already worked together with the VMware support, but they cant help. They suggest to revert to snap or restore from backup. Otherwise re-install side-by-side and manually migrate stuff. But that is not working

    As vCenter seems to run normally, I tried to do a backup through VAMI interface, but there it seems that it still recognizes the old u3n version and the backup fails ...

    This is some output from /var/log/vmware/applmgmt/backup.log


    I will attach the backup.log and software-packages.log to this thread. Maybe someone has an idea, how I can fix that half-updated state or create a backup so I can re-deploy vCenter from backup


    backup.txt   335 KB 1 version
    software-packages.log   786 KB 1 version

  • 2.  RE: Incomplete vCenter Update - Cant finalize it or revert
    Best Answer

    Broadcom Employee
    Posted Sep 25, 2021 03:22 AM

    2021-09-24T15:33:36.893 [ComponentScriptsBackup:PID-2493] ERROR: rc: 1, stderr: b'cat: write error: No space left on device\n'

    Seems u had space crunch. Fixing a broken update is really tough. Best is to revert back to snapshot before patching and then make sure you have enough space on root and other partitions even and then patch over new  snapshot

  • 3.  RE: Incomplete vCenter Update - Cant finalize it or revert

    Posted Sep 25, 2021 09:09 AM

    Yeah that space thing was taken from the backup.log, will check what exactly is running low on space.

    But I cant revert to snap or backup, cause there is none (or at least very old...).

    So I would have a chance if I can finalize that broken update or getting a clean configuration backup through VAMI so I can re-deploy with current data.

    That is my hope ;>

  • 4.  RE: Incomplete vCenter Update - Cant finalize it or revert

    Posted Sep 25, 2021 03:04 PM

    Good point with the storage

    I was able to successfully run a backup from the old environment after adding some space to the target backup space ... I just have overseen this

    Now, it created me a backup for 67u3n, so I re-deployed from backup with the corresponding vcsa full iso. Once the backup was restored, it failed at 70% of starting services. Not sure, what the problem was, as all services were online and vCenter accessible. When I try to login to VAMI, I always get redirected to https://vcsa.domain.tld:5480/configure/#/installer?locale=en with the message "vCenter Server has been restored. However, additional steps must be completed before it is available for use. Click the link below to continue.". Then asking for first root pw and then admin@vsphere.local password. So I provided both and it fails again while trying to start services.

    I tried running the upgrade to 67u3o, it was successfull. vCenter runs fine. Unfortunately, it still redirects me to this configure page...

    So I did another reboot after I followed the steps from this kb As the software_update_state.conf doesnt exist, I skipped that point and created the dir /var/log/vmware/upgrade and also those empty files, did a reboot, got redirected again, entered passwords and it finally went through successfully.

    So for now it is looking fine, at least to me ... will get in touch with VMware support again, to double check ...