VMware vSphere

 View Only
Expand all | Collapse all

Cannot login DOMAIN\ADMIN_USER@.... fills event log

  • 1.  Cannot login DOMAIN\ADMIN_USER@.... fills event log

    Posted Mar 03, 2011 02:24 PM

    Hi,

    I installed our vCenter a couple of weeks ago using a special domain admin account. Since then the vCenter is running without (known) issues. Today I had a look in the vCenter event log and found out that every 10 seconds a login attemp happens with the admin user I used during install. The @x.x.x.x IP adresse is the address of the vCenter host itself.   I'm connected with the vCenter Client with an non-admin account at the moment and have simply no idea which process is trying to access vCenter with the other user. I've rebooted the server last weeks (before I noticed this problem) which had no effect of the login attemps, they appear in the log files since many weeks.

    How can I find out which process is using this user? It must be configures somewhere, because the login errors start right after reboot. Maybe I configured the user and the password, which has changed since the install.

    I found a KB article. But it doen't help to find the process/programm which tries to login with the wrong user. I already know that it must be coming from the vCenter hosts IP, but I don't know where the user/password is stored.

    VirtualCenter logs contain numerous login failures

    http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006281



  • 2.  RE: Cannot login DOMAIN\ADMIN_USER@.... fills event log

    Posted Mar 03, 2011 02:32 PM

    Look under services (Run -> services.msc) and see if you can find the "old" login somewhere under Log On As

    Are there any other server which are running some sort of script or tool?



  • 3.  RE: Cannot login DOMAIN\ADMIN_USER@.... fills event log

    Posted Mar 03, 2011 02:40 PM

    Services are started as "Local System". The login attempts are coming from the vCenter host. I see successful logins from the nagios server, but this is a different IP and user. I also can't find process in the task manager that is running as the admin user.



  • 4.  RE: Cannot login DOMAIN\ADMIN_USER@.... fills event log

    Posted Mar 03, 2011 02:47 PM

    Check the following:-

    What service is the vcenter Service using to logon to ?

    is it a domain account ? if so is the account locked out in the domain ?

    Does the account have any admin rights ?

    Ensure that the system you use for your vCenter Server installation belongs to a domain rather than a workgroup.

    Ensure that the VirtualCenter Server service is using the right credentials. For example, if the credentials for the service belong to an account that only has Administrator privileges on a given domain, then the credentials must be changed to a domain account.

    To ensure VirtualCenter is using the right credentials:

    1. Click Start > Run, type services.msc, and press Enter.

    2. Double-click VMware VirtualCenter Server Service.

    3. Click the Log On tab.

    4. Select This Account, type the name of the account in the format of DOMAINNAME\accountname , or click Browse and find the correct account.

    5. Enter the appropriate password for the account, in the Password and Verify Password fields.

    6. Click OK.

    7. Start the service.

      Ensure no services use the vCenter Server ports. The default for VirtualCenter WebAccess is port 80. Stop any services that may be using this port.



  • 5.  RE: Cannot login DOMAIN\ADMIN_USER@.... fills event log

    Posted Mar 03, 2011 03:15 PM
    What service is the vcenter Service using to logon to ?

    Local System

    is it a domain account ? if so is the account locked out in the domain ?

    no, it uses the Local System

    I don't think changing "Log On As..." for the VirtualCenter Server service is the solution. vCenter is running fine, I don't see a problem with the vCenter service itself. There just must be something that tries to connect to the vCenter with wrong user/password. And it must be running on the same host.



  • 6.  RE: Cannot login DOMAIN\ADMIN_USER@.... fills event log

    Posted Mar 03, 2011 03:28 PM

    In Vcenter > Home> Management>Events>

    Is there anything specific in there ?

    I had the same issue where I had HP systems insight manager trying to connect to the vcenter with invalid password, and was getting loads of messages. It the vcenter event viewer it mentioned which host ip it was originating from



  • 7.  RE: Cannot login DOMAIN\ADMIN_USER@.... fills event log

    Posted Mar 03, 2011 04:01 PM

    from my first post:

    Today I had a look in the vCenter event log and found out that every 10 seconds a login attemp happens with the admin user I used during install. The @x.x.x.x IP adresse is the address of the vCenter host itself.

    I know which host tries to connect, the login attemps are comming from the vCenter host.

    I just rebooted the server and right after that the event logs starts to fill with these messages again.

    I've attached part of the event log.

    • EMEA\...._a_ralfgro is the admin Account that tries to login every 10 seconds
    • EMEA\ralfgro is my standard account, I'm connected with this account by RDP to the vCenter host and then with the vCenter Client and "Use Windows session credentials" to vCenter.

    You can see that I logged in successfull with the non admin account.



  • 8.  RE: Cannot login DOMAIN\ADMIN_USER@.... fills event log

    Posted Mar 04, 2011 07:30 AM

    I think this is the relevant part of the vpxd.log:

    [2011-03-04 08:05:05.134 04284 verbose 'SoapAdapter.HTTPService'] User agent is 'Java/1.6.0_18'
    [2011-03-04 08:05:05.134 04284 verbose 'SoapAdapter.HTTPService'] HTTP Response: Client: NeedsContentLength: false UnderstandsChunking: true CanKeepAlive: true (PresetContentLength -1)
    [2011-03-04 08:05:05.135 04284 verbose 'App'] [VpxVmomi] Invoking [login] on [vim.SessionManager:SessionManager] session [D530CFA5-EC87-4B3F-A1CA-AC7B53826FA1]
    [2011-03-04 08:05:05.136 04284 info 'App' opID=54ad2c19] [VpxLRO] -- BEGIN task-internal-13320 --  -- vim.SessionManager.login -- D530CFA5-EC87-4B3F-A1CA-AC7B53826FA1
    [2011-03-04 08:05:05.137 04284 info 'Libs' opID=54ad2c19] [ADS] Account xxxxx_a_ralfgro found, but not local
    [2011-03-04 08:05:05.138 04284 info 'App' opID=54ad2c19] Error 1326 authenticating user .\xxxxx_a_ralfgro.
    [2011-03-04 08:05:05.202 04284 info 'App' opID=54ad2c19] Error 1326 authenticating user EMEA\xxxx_a_ralfgro.
    [2011-03-04 08:05:05.202 04284 error 'App' opID=54ad2c19] Failed to authenticate user <EMEA\xxxx_a_ralfgro>
    [2011-03-04 08:05:05.203 04284 error 'App' opID=54ad2c19] vim.fault.InvalidLogin
    [2011-03-04 08:05:05.205 04284 verbose 'App' opID=54ad2c19] [VpxVmomi] Invoke error: vim.SessionManager.login session: D530CFA5-EC87-4B3F-A1CA-AC7B53826FA1 Throw: vim.fault.InvalidLogin
    [2011-03-04 08:05:05.205 04284 verbose 'SoapAdapter.HTTPService' opID=54ad2c19] User agent is 'Java/1.6.0_18'
    [2011-03-04 08:05:05.205 04284 verbose 'SoapAdapter.HTTPService' opID=54ad2c19] HTTP Response: Client: NeedsContentLength: false UnderstandsChunking: true CanKeepAlive: true (PresetContentLength -1)
    [2011-03-04 08:05:05.205 04284 verbose 'SoapAdapter.HTTPService' opID=54ad2c19] HTTP Response: Complete (processed 585 bytes)
    [2011-03-04 08:05:05.210 04284 info 'App' opID=54ad2c19] [VpxLRO] -- FINISH task-internal-13320 --  -- vim.SessionManager.login -- D530CFA5-EC87-4B3F-A1CA-AC7B53826FA1
    [2011-03-04 08:05:05.210 04284 info 'App' opID=54ad2c19] [VpxLRO] -- ERROR task-internal-13320 --  -- vim.SessionManager.login: vim.fault.InvalidLogin:
    Result:
    (vim.fault.InvalidLogin) {
       dynamicType = <unset>,
       faultCause = (vmodl.MethodFault) null,
       msg = "",
    }
    Args:
    Arg userName:
    "xxxxx_a_ralfgro"
    Arg password:
    (not shown)
    Arg locale:


  • 9.  RE: Cannot login DOMAIN\ADMIN_USER@.... fills event log



  • 10.  RE: Cannot login DOMAIN\ADMIN_USER@.... fills event log

    Posted Mar 03, 2011 03:00 PM

    Did you install the other extensions as well?  Converter/Update manager.  Are those working?

    -KjB



  • 11.  RE: Cannot login DOMAIN\ADMIN_USER@.... fills event log

    Posted Mar 03, 2011 03:20 PM

    Converter was installed, I uninstalled it this morning. Update Manager is installed and working. I've now disabled _all_ plugins.



  • 12.  RE: Cannot login DOMAIN\ADMIN_USER@.... fills event log

    Posted Mar 04, 2011 03:39 PM

    Disabling plugins are at a client level, not server.  Check which extensions are installed on your vcenter server.  Go into Programs, and remove those features, and/or reinstall them.  That should clear out any lingering extension that is not properly communicating with vcenter.

    A good reason here to use domain service accounts, and use them to register the add-ons, as opposed to an user admin account.

    -KjB



  • 13.  RE: Cannot login DOMAIN\ADMIN_USER@.... fills event log

    Posted Aug 24, 2011 04:44 PM

    For me, removing the HP Insight for Vcenter software stopped the login failures. It's been so long since I had installed it I didn't remember where to change the creds for it. I also did the lazy thing and used my account for the services to login with, which means that when I changed my password HP Insight started failing on login.

    So, I would focus right on that.    



  • 14.  RE: Cannot login DOMAIN\ADMIN_USER@.... fills event log

    Posted Sep 08, 2011 01:41 PM

    Hi.

    ralfgro did you solve your problem with "Cannot login" entries in vCenter Event Log?

    I've same problem, every 10 seconds some "userAgent java 1.6.0_18" tries to connect to vpxd.exe server with Administrator user account.

    Connections comes from vCenter host himself.

    The only one external plugin installed is

    Virtual Storage Console
    NetApp, Inc.
    2.0.1
    Virtual Storage Console for
    VMWare vSphere

    Please help.

    Update:

    I've fixed this issue by uninstalling Virtual Storage Console.

    It was registered with vCenter long time ago with domain admin account, someone changed domain admin password after that.

    Message was edited by: eLco



  • 15.  RE: Cannot login DOMAIN\ADMIN_USER@.... fills event log

    Posted Nov 03, 2011 01:16 PM

    I just had the same issue.  It turns out uninstalling NetApp's Virtual Storage Console 2.0.1P1 on the vCenter server resolved this issue.   I had installed it using the domain admistrator account.  I'm going to reinstall using a service account.

    :smileyhappy:



  • 16.  RE: Cannot login DOMAIN\ADMIN_USER@.... fills event log

    Posted Feb 20, 2012 08:05 PM

    I realize this is an old thread, but having recently run into a similar error that was somewhat tricky to solve, I thought would reply. I've run into this account lockout problem twice, under completely different circumstances, for my user account after updating my AD login password. The vpxd.log would show repeated entries similar to below (domain\user changed to protect the guilty :smileyhappy: ):

    [2012-02-20 08:46:49.367 04884 info 'App' opID=477a53e0] [VpxLRO] -- BEGIN task-internal-8441 --  -- vim.SessionManager.login -- 2E7A86B3-3689-4A70-AB39-F80CDD3047FD

    [2012-02-20 08:46:49.371 04884 info 'App' opID=477a53e0] Error 1909 authenticating user domain\user_account.
    [2012-02-20 08:46:49.371 04884 error 'App' opID=477a53e0] Failed to authenticate user <domain\user_account>
    [2012-02-20 08:46:49.371 04884 error 'App' opID=477a53e0] vim.fault.InvalidLogin
    The net result was a continual lockout of the AD account listed above. Using the Windows Resource kit tool eventcomb, we were able to look thru the AD controllers to pinpoint the offending source. Once we had that, we traced it back to one of our VirtualCenter boxes. After trying the usual troubleshooting, i.e. ensure no stale RDP/VC sessions, scheduled tasks, services set to Logon As, etc, there was nothing turning up. The only correlation was to Vcenter server-the account was only getting locked out when the vpxd service was running. After stopping the service and unlocking the account, it would remain unlocked until the vpxd service was restarted and then-bam! Account locked out again.
    __
    Long story short, there were two cuplrits at work. The first was VCO. We use it extensively in our environment. When you install and config various elements, you have to supply AD credentials. For the Update Mgr plugin config on one VCO server, it had been configured and then removed, but the VCO service had not been restarted-net result, the VCO server was attemtping to connect to the VC server every 30 seconds, and promptly locking out the user account with 4625 events recorded in the VCO server security log. Oddly enough, it was only until we reinstalled the plugin and restarted the service, did it stop this behavior. It almost seemed as if it got 'stuck' in this in-between state.
    __
    The second was with Quest vFoglight, specifically the Foglight VMware Collector service. This one was a bit tougher to figure out, as it didn't occur every 30 seconds like the VCO related issue. The agent was apparently installed some time ago and not removed but was not started until a recent system reboot, and it would periodically lock out the user account. Looking in the Quest collector.log file, you could see the following error:
    012-02-17 20:44:03,506 [7] INFO  Collector - GetInventory - Enter InventoryService.GetInventory
    2012-02-17 20:44:03,506 [6] INFO  Collector - GetFreshInventoryWorker - Enter InventoryService.GetFreshInventoryWorker
    2012-02-17 20:44:04,653 [6] ERROR Collector - GetFreshInventoryWorker - Error in InventoryService.GetFreshInventoryWorker: Vizioncore.Vmware.Vim.LoginFailedException: Could not log in to [https://10.10.10.1/sdk/vimService] as domain\user ---> System.Web.Services.Protocols.SoapException: Cannot complete login due to an incorrect user name or password.

    This would be generated each time the Foglight VMware Collector service was restarted, and it would also record the 1909 errors in the vpxd log.
    Two very different situations, but each very annoying. Since they were both connecting to the VC and then VC was authenticating to AD, made it a bit tougher to resolve then the typical AD lockout situations. Perhaps this will be of help to someone.


  • 17.  RE: Cannot login DOMAIN\ADMIN_USER@.... fills event log

    Posted Oct 25, 2012 05:56 PM

    dcraige27 wrote:

    I just had the same issue.  It turns out uninstalling NetApp's Virtual Storage Console 2.0.1P1 on the vCenter server resolved this issue.   I had installed it using the domain admistrator account.  I'm going to reinstall using a service account.

    :smileyhappy:

    This was precicely my error as well.

    I upgraded to VSC 4.0 using a service account and didn't encounter any more issues.



  • 18.  RE: Cannot login DOMAIN\ADMIN_USER@.... fills event log

    Posted Jan 10, 2013 11:44 AM

    dcraige27 wrote:

    I just had the same issue.  It turns out uninstalling NetApp's Virtual Storage Console 2.0.1P1 on the vCenter server resolved this issue.   I had installed it using the domain admistrator account.  I'm going to reinstall using a service account.

    :smileyhappy:

    This thread really helped. I have been scratching my head and jumping through hoops for a while trying to figure out where all of these failed logins were coming from. For me, it also turned out to be the NetApp Virtual Storage Console. Must have required a user account during install that has since been deleted. Thanks for the help!



  • 19.  RE: Cannot login DOMAIN\ADMIN_USER@.... fills event log

    Posted Apr 22, 2012 10:52 PM

    Solved.

    Check if you have Veeam monitor installed.

    If you do try stopping the Veeam Data Collector Service and see if the failures audits go away. If they do then change the login account credentials this service uses as appropriate. This resolved the issue for us after days of running into brick walls.