vSAN1

 View Only

  • 1.  vSAN - Update ESXi Configuration - Won't clear "vCenter state is authoratative"

    Posted Sep 02, 2021 01:11 AM

    Cluster after move and repair of VSAN has error of "Update ESXI Configuration"   for update ESXI Configuration.  All three hosts note : 

     

    Last Update by VC

    Different VC (60f584a0-1d04-3c42-154b-a0423f377a7e)

     

    Running wizard for vCenter to take over ownership of vSAN completes but never clears error.

     

    My assumption is the UUID "60f584a0-1d04-3c42-154b-a0423f377a7e"  was the UUID of the temp vCenter I used to repair the cluster.  But not sure how to get it to let go and revert under this vCenter.   Typical vCenter lack of detail logging to further root cause things

     

     



  • 2.  RE: vSAN - Update ESXi Configuration - Won't clear "vCenter state is authoratative"

    Posted Sep 08, 2021 09:01 PM

    <poke on this thread>

     

    Been working on other fires.. back to this topic:

     

    esxi_vCenter_not_Authoratative.png

     

    Back working on this project. 

     

    vCenter state is authoritativeSILENCE ALERT
    • Hosts that are out of sync
    • Info
    UPDATE ESXI CONFIGURATION

     

    Run update :  event task list shows success but hosts still listed as needing to be set to authoritative.

     

    tail /var/ syslog.log

    2021-09-08T20:33:21Z backup.sh[2381243]: Creating ConfigStore Backup
    2021-09-08T20:33:21Z configStoreBackup: ConfigStore backup completed successfully
    2021-09-08T20:33:21Z configStoreBackup: ConfigStore backup completed with rc = 101
    2021-09-08T20:33:21Z configStoreBackup: ConfigStore backup completed successfully
    2021-09-08T20:33:21Z configStoreBackup: ConfigStore backup completed with rc = 101
    2021-09-08T20:33:22.575Z ConfigStore[2381360]: Log for ConfigStore version=1.0 build=build-17867351 option=Release
    2021-09-08T20:33:22.575Z ConfigStore[2381360]: Could not expand environment variable HOME.
    2021-09-08T20:33:22.575Z ConfigStore[2381360]: Could not expand environment variable HOME.
    2021-09-08T20:33:22.575Z ConfigStore[2381360]: DictionaryLoad: Cannot open file "/usr/lib/vmware/config": No such file or directory.
    2021-09-08T20:33:22.575Z ConfigStore[2381360]: DictionaryLoad: Cannot open file "~/.vmware/config": No such file or directory.
    2021-09-08T20:33:22.575Z ConfigStore[2381360]: DictionaryLoad: Cannot open file "~/.vmware/preferences": No such file or directory.
    2021-09-08T20:33:22.575Z ConfigStore[2381360]: Switching to VMware syslog extensions
    2021-09-08T20:33:23Z backup.sh[2381243]: Locking esx.conf
    2021-09-08T20:33:23Z backup.sh[2381243]: Creating archive
    2021-09-08T20:33:23Z backup.sh[2381243]: Unlocking esx.conf
    2021-09-08T20:33:24Z backup.sh[2381243]: Using key ID 5c4c03bf-c118-48e3-a03c-1d34080191a3 to encrypt

    [root@thor:~] tail -f /var/log/vmkernel.log

    2021-09-08T20:59:00.173Z cpu25:2098895 opID=1afa4246)World: 11986: VC opID 11232485-W773-069e maps to vmkernel opID 1afa4246
    2021-09-08T20:59:00.173Z cpu25:2098895 opID=1afa4246)Config: 716: "ClomRepairDelay" = 60, Old Value: 60, (Status: 0x0)
    2021-09-08T20:59:00.189Z cpu25:2098895 opID=1afa4246)Config: 716: "DOMOwnerForceWarmCache" = 0, Old Value: 0, (Status: 0x0)
    2021-09-08T20:59:00.204Z cpu25:2098895 opID=1afa4246)Config: 716: "SwapThickProvisionDisabled" = 1, Old Value: 1, (Status: 0x0)
    2021-09-08T20:59:00.212Z cpu25:2098895 opID=1afa4246)Config: 716: "goto11" = 0, Old Value: 0, (Status: 0x0)
    2021-09-08T20:59:00.213Z cpu25:2098895 opID=1afa4246)Config: 716: "ClomBgProRebalanceEnabled" = 1, Old Value: 1, (Status: 0x0)
    2021-09-08T20:59:00.220Z cpu25:2098895 opID=1afa4246)Config: 716: "ClomBgProRebalanceThreshold" = 30, Old Value: 30, (Status: 0x0)
    2021-09-08T20:59:00.229Z cpu25:2098895 opID=1afa4246)Config: 716: "HostFailureThresholdState" = 0, Old Value: 0, (Status: 0x0)
    2021-09-08T20:59:00.233Z cpu25:2098895 opID=1afa4246)Config: 716: "InternalOpThresholdState" = 0, Old Value: 0, (Status: 0x0)
    2021-09-08T20:59:00.239Z cpu25:2098895 opID=1afa4246)RDT: RDTVSISetEnableRdma:2519: Rdma already disabled. Nothing to do.
    2021-09-08T20:59:00.242Z cpu25:2098895 opID=1afa4246)Config: 716: "DedupScope" = 0, Old Value: 0, (Status: 0x0)
    2021-09-08T20:59:00.250Z cpu25:2098895 opID=1afa4246)Config: 716: "GuestUnmap" = 0, Old Value: 0, (Status: 0x0)
    2021-09-08T20:59:00.253Z cpu25:2098895 opID=1afa4246)Config: 716: "DomCompResyncThrottle" = 0, Old Value: 0, (Status: 0x0)
    2021-09-08T20:59:00.565Z cpu26:2098902 opID=a511cdc6)World: 11986: VC opID 112324a5-06a5 maps to vmkernel opID a511cdc6
    2021-09-08T20:59:00.565Z cpu26:2098902 opID=a511cdc6)RDT: RDTVSIGetSubClusterSecCfgMode:4774: Current security mode 0, state 0
    2021-09-08T20:59:07.272Z cpu23:2097247)------------ ------------ ------------ ------------ ------------ ------------------------------
    2021-09-08T20:59:07.272Z cpu23:2097247) min,KB max,KB minLimit,KB eMin,KB rMinPeak,KB name
    2021-09-08T20:59:07.272Z cpu23:2097247)------------ ------------ ------------ ------------ ------------ ------------------------------
    2021-09-08T20:59:07.272Z cpu23:2097247) 204800 204800 -1 204800 204800 host/vim/vmvisor/config-file-tracker
    2021-09-08T20:59:07.272Z cpu23:2097247)------------ ------------ ------------ ------------ ------------ ------------------------------
    2021-09-08T20:59:07.272Z cpu23:2097247) 0 -1 -1 1092 72312 python.2383871
    2021-09-08T20:59:07.272Z cpu23:2097247) 0 -1 -1 132 132 uwWorldStore.2383871
    2021-09-08T20:59:07.272Z cpu23:2097247) 0 -1 -1 136 136 worldGroup.2383871
    2021-09-08T20:59:07.272Z cpu23:2097247) 0 -1 -1 0 70692 uw.2383871
    2021-09-08T20:59:07.272Z cpu23:2097247) 0 -1 -1 136 136 vsiHeap.2383871
    2021-09-08T20:59:07.272Z cpu23:2097247) 0 -1 -1 264 792 pt.2383871
    2021-09-08T20:59:07.272Z cpu23:2097247) 0 -1 -1 288 288 cartelheap.2383871
    2021-09-08T20:59:07.272Z cpu23:2097247) 0 -1 -1 0 0 uwshmempt.2383871
    2021-09-08T20:59:07.272Z cpu23:2097247) 0 -1 -1 136 136 uwAsyncRemapHeap.2383871
    2021-09-08T20:59:07.272Z cpu23:2097247)------------ ------------ ------------ ------------ ------------ ------------------------------
    2021-09-08T20:59:09.018Z cpu14:2098904 opID=aaad51e0)World: 11986: VC opID 112324fa-06b4 maps to vmkernel opID aaad51e0
    2021-09-08T20:59:09.018Z cpu14:2098904 opID=aaad51e0)RDT: RDTVSIGetSubClusterSecCfgMode:4774: Current security mode 0, state 0

    <<< not looking like much >>>

     

    Those are the outputs from log files from one server while I run vCenter upgrade



  • 3.  RE: vSAN - Update ESXi Configuration - Won't clear "vCenter state is authoratative"

    Posted Feb 14, 2022 07:18 PM

    < Update> 

     

    I put another new 1TB SSD into the server to see if it was something with the disk

     

    Task Name
     Add disks to the vSAN cluster
    Status
     A general system error occurred: Failed to reserve disk t10.ATA_____WDC__WDS100T2B0B2D00YS70_________________203612801631________ with exception: Failed to reserve disk t10.ATA_____WDC__WDS100T2B0B2D00YS70_________________203612801631________ with exception: Reserve failed with error code: -1
    Initiator
     com.vmware.vsan.health
     
     
    I tried to track down from other postings..  almost like the disk has some leftovers from previous vSAN on it and just can't figure out how to wipe disk 
     
     
    [root@odin:~] esxcfg-scsidevs -c |grep 183533804564
    t10.ATA_____WDC_WDS100T2B0B2D00YS70__________________183533804564________ Direct-Access /vmfs/devices/disks/t10.ATA_____WDC_WDS100T2B0B2D00YS70__________________183533804564________ 953869MB HPP Local ATA Disk (t10.ATA_____WDC_WDS100T2B0B2D00YS70__________________183533804564________)
    [root@odin:~] esxcfg-scsidevs -ld t10.ATA_____WDC_WDS100T2B0B2D00YS70__________________183533804564________
    t10.ATA_____WDC_WDS100T2B0B2D00YS70__________________183533804564________
    Device Type: Direct-Access
    Size: 953869 MB
    Display Name: Local ATA Disk (t10.ATA_____WDC_WDS100T2B0B2D00YS70__________________183533804564________)
    Multipath Plugin: HPP
    Console Device: /vmfs/devices/disks/t10.ATA_____WDC_WDS100T2B0B2D00YS70__________________183533804564________
    Devfs Path: /vmfs/devices/disks/t10.ATA_____WDC_WDS100T2B0B2D00YS70__________________183533804564________
    Vendor: ATA Model: WDC WDS100T2B0B- Revis: 90WD
    SCSI Level: 5 Is Pseudo: false Status: on
    Is RDM Capable: false Is Removable: false
    Is Local: true Is SSD: true
    Other Names:
    vml.01000000003138333533333830343536342020202020202020574443205744
    VAAI Status: unsupported
    [root@odin:~] partedUtil getptbl /vmfs/devices/disks/t10.ATA_____WDC_WDS100T2B0B2D00YS70__________________183533804564________
    msdos
    121601 255 63 1953525168

    [root@odin:~] esxcfg-mpath -ld t10.ATA_____WDC_WDS100T2B0B2D00YS70__________________183533804564________
    sata.vmhba0-sata.0:2-t10.ATA_____WDC_WDS100T2B0B2D00YS70__________________183533804564________
    Runtime Name: vmhba0:C0:T2:L0
    Device: t10.ATA_____WDC_WDS100T2B0B2D00YS70__________________183533804564________
    Device Display Name: Local ATA Disk (t10.ATA_____WDC_WDS100T2B0B2D00YS70__________________183533804564________)
    Adapter: vmhba0 Channel: 0 Target: 2 LUN: 0
    Adapter Identifier: sata.vmhba0
    Target Identifier: sata.0:2
    Plugin: HPP
    State: active
    Transport: sata

    [root@odin:~] dd if=/dev/zero of=/vmfs/devices/disks/t10.ATA_____WDC_WDS100T2B0B2D00YS70__________________183533804564________

    dd: can't open '/vmfs/devices/disks/t10.ATA_____WDC_WDS100T2B0B2D00YS70__________________183533804564________': Function not implemented
    [root@odin:~]


    But .. back to esxi hamstrug from doing lower level wipefs or dd etc.. to remove data from disk
     
     
    I have another server .. exact same motherboard, RAID controller,  firmware,  disk drives,  which is working without issue.  So it has to be some kind of configuration delta.