VMware vSphere

 View Only

  • 1.  vCenter appliance CPU Usage

    Posted Jan 28, 2013 03:31 PM

    hi guys

    I have an instance of vCenter appliance build 799731.

    I see how the CPU Usage have been increasing in the last month and I get alerts about that (we have configure an open source alert system)

    check this out

    Initially cpu percentage 27% - 12/29/2012 and as of today 75%.

    Memory 8Gb, 2 vCPUs

    any idea what could be causing this?

    We have right now only machines for testing around 25 VMs nothing is in producción and we have HA-DRS-sDRS configured

    thanks a lot



  • 2.  RE: vCenter appliance CPU Usage

    Posted Jan 29, 2013 02:48 AM

    I found vpxd is always 100% - 99% cpu usage

    anyone has seem this?

    by the way no one is affected we can login without any problem and check everythign is only the vpxd usage is pretty high all the time

    I am really afraid when this will be producction and 100 - 150 VMs

    ----------------------------------------

    update

    I was checking the vpxd-###.log

    I found this repeats like every 10 seconds, I really don't get what is this active session is doing I mean this user I can see in vCenter GUI in sessions how this username updates itselft like 10 seconds, it happens at the same time username shows up in the log username updates in the session GUI

    2013-01-29T04:28:22.823Z [7F911BD34700 info '[SSO]' opID=10b7ca39] [UserDirectorySso] Authenticate(jcastegnaro, "not shown")
    2013-01-29T04:28:22.869Z [7F911BD34700 info '[SSO]' opID=10b7ca39] User MYCOMANYDOMAIN\USERNAME authenticated successfully.
    2013-01-29T04:28:22.870Z [7F911BD34700 info 'Default' opID=10b7ca39] [Auth]: User MYCOMANYDOMAIN\USERNAME
    2013-01-29T04:28:22.872Z [7F911BD34700 info 'commonvpxLro' opID=10b7ca39] [VpxLRO] -- FINISH task-internal-839625 --  -- vim.SessionManager.login --
    2013-01-29T04:28:23.790Z [7F9122656700 info 'commonvpxLro' opID=60974493] [VpxLRO] -- BEGIN task-internal-839626 --  -- vim.event.EventManager.createCollector -- 0 d5a6ac7-4042-b61f-4836-3cf6cd04428a(5233b643-106c-ff6c-33e4-2d149ee37428)
    2013-01-29T04:28:26.967Z [7F9122656700 warning 'Default' opID=60974493] [VdbStatement] SQL execution took too long: SELECT EVENT_ID, CHAIN_ID, EVENT_TYPE, EXTENDED _CLASS, CREATE_TIME, USERNAME, CATEGORY, VM_ID, VM_NAME, HOST_ID, HOST_NAME, COMPUTERESOURCE_ID, COMPUTERESOURCE_TYPE, COMPUTERESOURCE_NAME, DATACENTER_ID, DATACEN TER_NAME, DATASTORE_ID, DATASTORE_NAME, NETWORK_ID, NETWORK_NAME, NETWORK_TYPE, DVS_ID, DVS_NAME, STORAGEPOD_ID, STORAGEPOD_NAME, CHANGE_TAG_ID FROM VPX_EVENT WHER E CREATE_TIME>=? AND EVENT_ID<? ORDER BY EVENT_ID DESC
    2013-01-29T04:28:26.967Z [7F9122656700 warning 'Default' opID=60974493] [VdbStatement] Execution elapsed time: 3176 ms
    2013-01-29T04:28:26.967Z [7F9122656700 warning 'Default' opID=60974493] [VdbStatement] Bind parameters:
    2013-01-29T04:28:26.967Z [7F9122656700 warning 'Default' opID=60974493] [VdbStatement] datatype: 10, size: 23, arraySize: 0
    2013-01-29T04:28:26.967Z [7F9122656700 warning 'Default' opID=60974493] [VdbStatement] datatype: 7, size: 10, arraySize: 0
    2013-01-29T04:28:26.967Z [7F9122656700 warning 'Default' opID=60974493] [VdbStatement] value = "2147483647"
    2013-01-29T04:28:27.199Z [7F9122656700 info 'commonvpxLro' opID=60974493] [VpxLRO] -- FINISH task-internal-839626 --  -- vim.event.EventManager.createCollector --
    2013-01-29T04:28:27.251Z [7F911BEBA700 info 'commonvpxLro' opID=d6c3c3c] [VpxLRO] -- BEGIN task-internal-839627 --  -- vim.event.EventHistoryCollector.readNext --  0d5a6ac7-4042-b61f-4836-3cf6cd04428a(5233b643-106c-ff6c-33e4-2d149ee37428)
    2013-01-29T04:28:27.252Z [7F911BAAA700 info 'commonvpxLro' opID=fd2ec26c] [VpxLRO] -- BEGIN task-internal-839628 --  -- vmodl.query.PropertyCollector.retrieveConte nts -- 336d4f23-e477-85e4-319b-8c15cc3a974a(52623936-9f84-53f8-3b2d-ca67bc2805c6)
    2013-01-29T04:28:27.252Z [7F911BAAA700 info 'commonvpxLro' opID=fd2ec26c] [VpxLRO] -- FINISH task-internal-839628 --  -- vmodl.query.PropertyCollector.retrieveCont ents --
    2013-01-29T04:28:27.254Z [7F9120B2C700 info 'commonvpxLro' opID=3e5eb169] [VpxLRO] -- BEGIN task-internal-839629 --  -- vmodl.query.PropertyCollector.retrieveConte nts -- 336d4f23-e477-85e4-319b-8c15cc3a974a(52623936-9f84-53f8-3b2d-ca67bc2805c6)
    2013-01-29T04:28:27.254Z [7F9120B2C700 info 'commonvpxLro' opID=3e5eb169] [VpxLRO] -- FINISH task-internal-839629 --  -- vmodl.query.PropertyCollector.retrieveCont ents --
    2013-01-29T04:28:28.680Z [7F911BAEB700 info 'commonvpxLro' opID=352f680a] [VpxLRO] -- BEGIN task-internal-839630 --  -- vim.TaskManager.createCollector -- 336d4f23 -e477-85e4-319b-8c15cc3a974a(52623936-9f84-53f8-3b2d-ca67bc2805c6)
    2013-01-29T04:28:28.686Z [7F911BAEB700 info 'commonvpxLro' opID=352f680a] [VpxLRO] -- FINISH task-internal-839630 --  -- vim.TaskManager.createCollector --
    2013-01-29T04:28:28.688Z [7F911BFBE700 info 'commonvpxLro' opID=c64bf90d] [VpxLRO] -- BEGIN task-internal-839631 --  -- vim.HistoryCollector.setLatestPageSize -- 3 36d4f23-e477-85e4-319b-8c15cc3a974a(52623936-9f84-53f8-3b2d-ca67bc2805c6)
    2013-01-29T04:28:28.690Z [7F911BFBE700 info 'commonvpxLro' opID=c64bf90d] [VpxLRO] -- FINISH task-internal-839631 --  -- vim.HistoryCollector.setLatestPageSize --
    2013-01-29T04:28:28.691Z [7F911BD34700 info 'commonvpxLro' opID=a7199482] [VpxLRO] -- BEGIN task-internal-839632 --  -- vmodl.query.PropertyCollector.retrieveConte nts -- 336d4f23-e477-85e4-319b-8c15cc3a974a(52623936-9f84-53f8-3b2d-ca67bc2805c6)
    2013-01-29T04:28:28.692Z [7F911BD34700 info 'commonvpxLro' opID=a7199482] [VpxLRO] -- FINISH task-internal-839632 --  -- vmodl.query.PropertyCollector.retrieveCont ents --
    2013-01-29T04:28:28.696Z [7F9120E90700 info 'commonvpxLro' opID=2aad6e3] [VpxLRO] -- BEGIN task-internal-839633 --  -- vim.TaskHistoryCollector.readNext -- 336d4f2 3-e477-85e4-319b-8c15cc3a974a(52623936-9f84-53f8-3b2d-ca67bc2805c6)
    2013-01-29T04:28:28.701Z [7F9120E90700 info 'commonvpxLro' opID=2aad6e3] [VpxLRO] -- FINISH task-internal-839633 --  -- vim.TaskHistoryCollector.readNext --
    2013-01-29T04:28:28.710Z [7F91209E7700 info 'commonvpxLro' opID=e206f520] [VpxLRO] -- BEGIN task-internal-839634 --  -- vim.TaskHistoryCollector.readNext -- 336d4f 23-e477-85e4-319b-8c15cc3a974a(52623936-9f84-53f8-3b2d-ca67bc2805c6)
    2013-01-29T04:28:28.711Z [7F91209E7700 info 'commonvpxLro' opID=e206f520] [VpxLRO] -- FINISH task-internal-839634 --  -- vim.TaskHistoryCollector.readNext --
    2013-01-29T04:28:28.712Z [7F911BF7D700 info 'commonvpxLro' opID=a8ce72de] [VpxLRO] -- BEGIN task-internal-839635 --  -- vim.HistoryCollector.remove -- 336d4f23-e47 7-85e4-319b-8c15cc3a974a(52623936-9f84-53f8-3b2d-ca67bc2805c6)
    2013-01-29T04:28:28.712Z [7F911BF7D700 info 'commonvpxLro' opID=a8ce72de] [VpxLRO] -- FINISH task-internal-839635 --  -- vim.HistoryCollector.remove --
    2013-01-29T04:28:28.714Z [7F91206DB700 info 'commonvpxLro' opID=51021113] [VpxLRO] -- BEGIN task-internal-839636 --  -- vmodl.query.PropertyCollector.retrieveConte nts -- 336d4f23-e477-85e4-319b-8c15cc3a974a(52623936-9f84-53f8-3b2d-ca67bc2805c6)
    2013-01-29T04:28:28.714Z [7F91206DB700 info 'commonvpxLro' opID=51021113] [VpxLRO] -- FINISH task-internal-839636 --  -- vmodl.query.PropertyCollector.retrieveCont ents --
    2013-01-29T04:28:28.715Z [7F911BE38700 info 'commonvpxLro' opID=5a6febe8] [VpxLRO] -- BEGIN task-internal-839637 --  -- vmodl.query.PropertyCollector.retrieveConte nts -- 336d4f23-e477-85e4-319b-8c15cc3a974a(52623936-9f84-53f8-3b2d-ca67bc2805c6)
    2013-01-29T04:28:28.716Z [7F911BE38700 info 'commonvpxLro' opID=5a6febe8] [VpxLRO] -- FINISH task-internal-839637 --  -- vmodl.query.PropertyCollector.retrieveCont ents --
    2013-01-29T04:28:28.717Z [7F91207DF700 info 'commonvpxLro' opID=f35cd9c0] [VpxLRO] -- BEGIN task-internal-839638 --  -- vmodl.query.PropertyCollector.retrieveConte nts -- 336d4f23-e477-85e4-319b-8c15cc3a974a(52623936-9f84-53f8-3b2d-ca67bc2805c6)
    2013-01-29T04:28:28.717Z [7F91207DF700 info 'commonvpxLro' opID=f35cd9c0] [VpxLRO] -- FINISH task-internal-839638 --  -- vmodl.query.PropertyCollector.retrieveCont ents --
    2013-01-29T04:28:28.719Z [7F91217D0700 info 'commonvpxLro' opID=d15fb13a] [VpxLRO] -- BEGIN task-internal-839639 --  -- vmodl.query.PropertyCollector.retrieveConte nts -- 336d4f23-e477-85e4-319b-8c15cc3a974a(52623936-9f84-53f8-3b2d-ca67bc2805c6)
    2013-01-29T04:28:28.719Z [7F91217D0700 info 'commonvpxLro' opID=d15fb13a] [VpxLRO] -- FINISH task-internal-839639 --  -- vmodl.query.PropertyCollector.retrieveCont ents --
    2013-01-29T04:28:28.829Z [7F911BFBE700 info 'commonvpxLro' opID=5a720587] [VpxLRO] -- BEGIN task-internal-839640 --  -- vmodl.query.PropertyCollector.retrieveConte nts -- 336d4f23-e477-85e4-319b-8c15cc3a974a(52623936-9f84-53f8-3b2d-ca67bc2805c6)
    2013-01-29T04:28:28.830Z [7F911BFBE700 info 'commonvpxLro' opID=5a720587] [VpxLRO] -- FINISH task-internal-839640 --  -- vmodl.query.PropertyCollector.retrieveCont ents --
    2013-01-29T04:28:28.831Z [7F91207DF700 info 'commonvpxLro' opID=cc375acc] [VpxLRO] -- BEGIN task-internal-839641 --  -- vmodl.query.PropertyCollector.retrieveConte nts -- 336d4f23-e477-85e4-319b-8c15cc3a974a(52623936-9f84-53f8-3b2d-ca67bc2805c6)
    2013-01-29T04:28:28.832Z [7F91207DF700 info 'commonvpxLro' opID=cc375acc] [VpxLRO] -- FINISH task-internal-839641 --  -- vmodl.query.PropertyCollector.retrieveCont ents --
    2013-01-29T04:28:30.477Z [7F911BEBA700 warning 'Default' opID=d6c3c3c] [VdbStatement] SQL execution took too long: SELECT EVENT_ID, CHAIN_ID, EVENT_TYPE, EXTENDED_ CLASS, CREATE_TIME, USERNAME, CATEGORY, VM_ID, VM_NAME, HOST_ID, HOST_NAME, COMPUTERESOURCE_ID, COMPUTERESOURCE_TYPE, COMPUTERESOURCE_NAME, DATACENTER_ID, DATACENT ER_NAME, DATASTORE_ID, DATASTORE_NAME, NETWORK_ID, NETWORK_NAME, NETWORK_TYPE, DVS_ID, DVS_NAME, STORAGEPOD_ID, STORAGEPOD_NAME, CHANGE_TAG_ID FROM VPX_EVENT WHERE  CREATE_TIME>=? AND EVENT_ID>? ORDER BY EVENT_ID ASC
    2013-01-29T04:28:30.477Z [7F911BEBA700 warning 'Default' opID=d6c3c3c] [VdbStatement] Execution elapsed time: 3226 ms
    2013-01-29T04:28:30.477Z [7F911BEBA700 warning 'Default' opID=d6c3c3c] [VdbStatement] Bind parameters:
    2013-01-29T04:28:30.477Z [7F911BEBA700 warning 'Default' opID=d6c3c3c] [VdbStatement] datatype: 10, size: 23, arraySize: 0
    2013-01-29T04:28:30.477Z [7F911BEBA700 warning 'Default' opID=d6c3c3c] [VdbStatement] datatype: 7, size: 1, arraySize: 0
    2013-01-29T04:28:30.477Z [7F911BEBA700 warning 'Default' opID=d6c3c3c] [VdbStatement] value = "0"
    2013-01-29T04:28:30.730Z [7F9120C30700 info 'commonvpxLro' opID=bc218292] [VpxLRO] -- BEGIN task-internal-839642 --  -- vim.ServiceInstance.GetServerClock -- bcb68 db0-8e32-353b-7e31-0b38daedcb34(52b19a1e-e127-7413-d368-93ce3ee568bf)
    2013-01-29T04:28:30.730Z [7F9120C30700 info 'commonvpxLro' opID=bc218292] [VpxLRO] -- FINISH task-internal-839642 --  -- vim.ServiceInstance.GetServerClock --
    2013-01-29T04:28:30.811Z [7F911BEBA700 info 'commonvpxLro' opID=d6c3c3c] [VpxLRO] -- FINISH task-internal-839627 --  -- vim.event.EventHistoryCollector.readNext --
    2013-01-29T04:28:30.847Z [7F911BF7D700 info 'commonvpxLro' opID=67f179f6] [VpxLRO] -- BEGIN task-internal-839643 --  -- vim.SessionManager.logout -- 0d5a6ac7-4042- b61f-4836-3cf6cd04428a(5233b643-106c-ff6c-33e4-2d149ee37428)
    2013-01-29T04:28:30.847Z [7F911BF7D700 info 'commonvpxLro' opID=67f179f6] [VpxLRO] -- FINISH task-internal-839643 --  -- vim.SessionManager.logout --
    2013-01-29T04:28:30.880Z [7F911BEBA700 info 'commonvpxLro' opID=3632ae33] [VpxLRO] -- BEGIN task-internal-839644 --  -- vim.ServiceInstance.retrieveContent -- 9b96 3cce-196e-4aa7-3548-8a9122f6763b
    2013-01-29T04:28:30.881Z [7F911BEBA700 info 'commonvpxLro' opID=3632ae33] [VpxLRO] -- FINISH task-internal-839644 --  -- vim.ServiceInstance.retrieveContent --
    2013-01-29T04:28:30.883Z [7F911BBAE700 info 'commonvpxLro' opID=a96775ac] [VpxLRO] -- BEGIN task-internal-839645 --  -- vim.ServiceInstance.retrieveContent -- 9b96 3cce-196e-4aa7-3548-8a9122f6763b
    2013-01-29T04:28:30.883Z [7F911BBAE700 info 'commonvpxLro' opID=a96775ac] [VpxLRO] -- FINISH task-internal-839645 --  -- vim.ServiceInstance.retrieveContent --
    2013-01-29T04:28:30.885Z [7F9120C30700 info 'commonvpxLro' opID=44694be3] [VpxLRO] -- BEGIN task-internal-839646 --  -- vim.SessionManager.login -- 9b963cce-196e-4 aa7-3548-8a9122f6763b

    thanks a lot



  • 3.  RE: vCenter appliance CPU Usage

    Posted Jan 29, 2013 10:52 PM

    guys we found that vpxd is using the CPU at 100% most of the time.

    and also found that vmware data protection is the cause for this

    We are opening a case with vmware



  • 4.  RE: vCenter appliance CPU Usage

    Posted Dec 05, 2013 05:38 PM

    Seeing similar behavior. Did you get this resolved?



  • 5.  RE: vCenter appliance CPU Usage

    Posted Dec 06, 2013 08:01 AM

    Which build is this?

    Regards

    Girish



  • 6.  RE: vCenter appliance CPU Usage

    Posted Dec 06, 2013 12:59 PM

    VDP 5.5.5



  • 7.  RE: vCenter appliance CPU Usage

    Posted Feb 12, 2014 02:12 PM

    Also interested by this: (I have the same exact problem)

    VCSA  5.5.0 b (5.5.0.5201 Build 1476389)

    VDP version 5.5.5.180



  • 8.  RE: vCenter appliance CPU Usage

    Posted Feb 17, 2014 12:49 PM

    Same problem here : excessive login/logout of VDP to vcenter increase vpx_event table size.

    A simple workaround is to reduce events retention policy to 1 day on vCenter Server Settings and wait few hour the cleanup task.