Dagent's falling off the network

[Migration User]

Hello

 

we are expereincing problems with dagent client/server communications when imaging machines. The imaging job goes ok; the client pxe boots and pulls the image down. Sysprep runs and the machine will change its name and join the domain. The only other job to do then is install AV (sophos) but after rebooting when it joins the domain the machine never comes back online in the console. the client log shows the following...



[06/15/2011 00:18:33.084 3704 0] bootstrap.cpp:450 GetCfgFilename(): strName = C:
[06/15/2011 00:18:38.787 2532 0] ReplyUpdateComputer.cpp:171 - Error removing registry value.
[06/15/2011 00:18:38.787 2532 0] ReplyUpdateComputer.cpp:247 - Error reading logevent status from registry.
[06/15/2011 00:20:15.458 3768 0] UiPipeListenerThread.cpp:77 - CUiPipeListener::Run caught CIPCError exception, removing UI from List.
[06/15/2011 00:22:21.681 1184 0] UiPipeListenerThread.cpp:77 - CUiPipeListener::Run caught CIPCError exception, removing UI from List.
[06/15/2011 00:23:47.950 3088 0] UiPipeListenerThread.cpp:77 - CUiPipeListener::Run caught CIPCError exception, removing UI from List.
[06/15/2011 00:25:47.135 3972 0] UiPipeListenerThread.cpp:77 - CUiPipeListener::Run caught CIPCError exception, removing UI from List.
[06/15/2011 00:37:27.770 2692 0] UiPipeListenerThread.cpp:77 - CUiPipeListener::Run caught CIPCError exception, removing UI from List.
[06/15/2011 00:41:7.864 3312 0] UiPipeListenerThread.cpp:77 - CUiPipeListener::Run caught CIPCError exception, removing UI from List.
[06/15/2011 04:44:11.552 2220 0] UiPipeListenerThread.cpp:77 - CUiPipeListener::Run caught CIPCError exception, removing UI from List.
[06/15/2011 05:07:36.788 2484 0] UiInitialConnectThread.cpp:97 - CUIError error, CUiInitialConnect::ReadConnectPacket() - Shutting down. - 0
[06/15/2011 05:07:36.788 2484 0] UiInitialConnectThread.cpp:138 - CUiInitialConnect::Run RETURNING...
[06/15/2011 05:07:36.819 1996 0] apps\dsagent\agent\LogEventThread.cpp:57 - CLogEventThread::Run CLogEventError
    Thrown from apps\dsagent\agent\LogEventThread.cpp:113 -message = CLogEventThread::ReadLogEvent() - Shutting down.
    SystemError Value = 0x00000000
    
[06/15/2011 05:07:36.819 3704 0] DsAgent.cpp:392 About to serialize objects
[06/15/2011 05:07:36.819 3716 0] main.cpp:780 - Leaving Main()
[06/15/2011 05:07:37.849 2644 0] bootstrap.cpp:1114 - Error reading input file.
[06/15/2011 05:07:38.223 2644 0] bootstrap.cpp:450 GetCfgFilename(): strName = C:
[06/15/2011 05:07:41.561 2076 0] ReplyUpdateComputer.cpp:171 - Error removing registry value.
[06/15/2011 05:07:41.561 2076 0] ReplyUpdateComputer.cpp:247 - Error reading logevent status from registry.
[06/15/2011 05:08:19.747 4024 0] UiInitialConnectThread.cpp:97 - CUIError error, CUiInitialConnect::ReadConnectPacket() - Shutting down. - 0
[06/15/2011 05:08:19.747 4024 0] UiInitialConnectThread.cpp:138 - CUiInitialConnect::Run RETURNING...
[06/15/2011 05:08:23.008 3568 0] UiPipeListenerThread.cpp:77 - CUiPipeListener::Run caught CIPCError exception, removing UI from List.
[06/15/2011 05:08:23.008 136 0] apps\dsagent\agent\LogEventThread.cpp:57 - CLogEventThread::Run CLogEventError
    Thrown from apps\dsagent\agent\LogEventThread.cpp:113 -message = CLogEventThread::ReadLogEvent() - Shutting down.
    SystemError Value = 0x00000000
    
[06/15/2011 05:08:23.008 2644 0] DsAgent.cpp:392 About to serialize objects
[06/15/2011 05:08:23.008 3680 0] main.cpp:780 - Leaving Main()

-----------------

The problem appears to be limited to 64bit Windows7 installs.

To troubleshoot I stop the dagent processes from running on the client and remove it from add/remove programs
I disabled the firewall
I confirm UAC is set to "never notify"
I reinstall the dagent

The re-install process says the install completed successfully....


Install settings:
Connecting to server x on port 402.
This agent can be remote controlled from the console.
The agent is password protected.
Prompting for execution, reboot, and/or remote control events.
The user will be prompted for 30 seconds, then the operation will continue.
Always connected to the server, refreshing every 28800 seconds.

Events:
Starting install for machine client-61d.
Connecting to machine client-61d.
Successfully connected to machine.
Installing to directory C:\Program Files\Altiris\AClient\.
Copying Agent program files to machine.
Agent successfully copied.
The install completed successfully.
--------------------------------------------------

...but the machine is still listed as offline in the console.

Logs from the client machine are below


[08/01/2011 06:24:0.602 724 0] DsAgent.cpp:392 About to serialize objects
[08/01/2011 06:24:1.632 724 0] bootstrap.cpp:1114 - Error reading input file.
[08/01/2011 06:24:1.773 724 0] bootstrap.cpp:450 GetCfgFilename(): strName = C:
[08/01/2011 06:24:5.551 1360 0] UiInitialConnectThread.cpp:97 - CUIError error, CUiInitialConnect::ReadConnectPacket() - Shutting down. - 0
[08/01/2011 06:24:5.551 1360 0] UiInitialConnectThread.cpp:138 - CUiInitialConnect::Run RETURNING...
[08/01/2011 06:24:5.582 844 0] apps\dsagent\agent\LogEventThread.cpp:57 - CLogEventThread::Run CLogEventError
    Thrown from apps\dsagent\agent\LogEventThread.cpp:113 -message = CLogEventThread::ReadLogEvent() - Shutting down.
    SystemError Value = 0x00000000
    
[08/01/2011 06:24:5.582 724 0] DsAgent.cpp:392 About to serialize objects
[08/01/2011 06:24:6.612 724 0] bootstrap.cpp:1114 - Error reading input file.
[08/01/2011 06:24:6.737 724 0] bootstrap.cpp:450 GetCfgFilename(): strName = C:
[08/01/2011 06:24:18.835 2596 0] UiInitialConnectThread.cpp:113 - CIPCError error connecting, CWinPipeIPC::Write  - 6
[08/01/2011 06:24:18.835 2596 0] UiInitialConnectThread.cpp:138 - CUiInitialConnect::Run RETURNING...
[08/01/2011 06:24:20.303 644 0] apps\dsagent\agent\LogEventThread.cpp:57 - CLogEventThread::Run CLogEventError
    Thrown from apps\dsagent\agent\LogEventThread.cpp:113 -message = CLogEventThread::ReadLogEvent() - Shutting down.
    SystemError Value = 0x00000000
 
    
[08/01/2011 06:24:24.721 724 0] DsAgent.cpp:392 About to serialize objects
[08/01/2011 06:24:25.751 724 0] bootstrap.cpp:1114 - Error reading input file.
[08/01/2011 06:24:25.923 724 0] bootstrap.cpp:450 GetCfgFilename(): strName = C:
[08/01/2011 06:24:25.923 3456 0] UiInitialConnectThread.cpp:49 - CIPCError
    Thrown from common\ipc\WinPipeIpc.cpp:286 -message = CWinPipeIPC::Listen() INVALID_HANDLE
    SystemError Value = 0x000000E7

------------------------------------------------------

This issue can be caused by the AV on the server (there is no AV on the client at this stage) treating named pipes as files. Disabling the AV on the server seems to have had no effect on the situation

The server is Win2003 Standard running deploy 6.9 SP4

Any other ideas?

[Migration User]

I too have basically the same problem; notably the CWinPipeIPC::Listen() INVALID_HANDLE part.

This is also for a Windows 7 64-bit Enterprise build. However, I'd also like to note that this is a new issue for me. With no changes to the image, this issue appears to have cropped up in relation to Windows Updates.

So, I rolled back about 32 Windows Updates from my deployment server (since a whole bunch got installed in one go), and that's resolved my issue. I suggest looking at rolling back recent Windows Updates on your server as well until the agent starts connecting again.

I just rolled back them all in one go and then restarted, so I don't know which one caused the problem...

[Migration User]

I hope it isn't the only soloution as i don't think i'd get away with it here...

[Migration User]

I thought I had resolved this by turning off on-access scanning on the AV on the server. it was fine for a few weeks but is now back.

it seems to only effect Win7 64 bit clients.

if I go to the Win7 machine -> DaggentConfig.exe -> client settings -> security and turn off "Encrypt session communication with Deployment Server" the machine will show up in the console. turn it back on again and it goes offline in the console.

With encryption off the coonection is successful and I get this in the client log...

bootstrap.cpp:1114 - Error reading input file.
bootstrap.cpp:450 GetCfgFilename(): strName = C:
ReplyUpdateComputer.cpp:171 - Error removing registry value.
ReplyUpdateComputer.cpp:247 - Error reading logevent status from registry.
UiInitialConnectThread.cpp:97 - CUIError error, CUiInitialConnect::ReadConnectPacket() - Shutting down. - 0
UiInitialConnectThread.cpp:138 - CUiInitialConnect::Run RETURNING...
apps\dsagent\agent\LogEventThread.cpp:57 - CLogEventThread::Run CLogEventError
    Thrown from apps\dsagent\agent\LogEventThread.cpp:113 -message =    CLogEventThread::ReadLogEvent() - Shutting down.
    SystemError Value = 0x00000000
    
DsAgent.cpp:392 About to serialize objects
main.cpp:780 - Leaving Main()

---------------------------------------------------------------------------------------

With encryption on the connection fails and I get this in the client log

DsAgent.cpp:392 About to serialize objects
bootstrap.cpp:1114 - Error reading input file.
bootstrap.cpp:450 GetCfgFilename(): strName = C:
UiInitialConnectThread.cpp:97 - CUIError error, CUiInitialConnect::ReadConnectPacket() - Shutting down. - 0
UiInitialConnectThread.cpp:138 - CUiInitialConnect::Run RETURNING...
apps\dsagent\agent\LogEventThread.cpp:57 - CLogEventThread::Run CLogEventError
    Thrown from apps\dsagent\agent\LogEventThread.cpp:113 -message = CLogEventThread::ReadLogEvent() - Shutting down.
    SystemError Value = 0x00000000

 

DsAgent.cpp:392 About to serialize objects
bootstrap.cpp:1114 - Error reading input file.
bootstrap.cpp:450 GetCfgFilename(): strName = C:
UiInitialConnectThread.cpp:49 - CIPCError
    Thrown from common\ipc\WinPipeIpc.cpp:286 -message = CWinPipeIPC::Listen() INVALID_HANDLE
    SystemError Value = 0x000000E7

[Migration User]

I am seeing the following.  Currently running DS 6.9SP3.  What version are you all on?  These errors are causing memory spikes on our servers and they essentially will then become unsuable and require a reboot to clear up the issues.

 

[09/11/2011 21:09:43.296 286708 0] bootstrap.cpp:445 GetCfgFilename(): strName = C:
[09/11/2011 21:09:43.296 38244 0] UiInitialConnectThread.cpp:49 - CIPCError
 Thrown from common\ipc\WinPipeIpc.cpp:286 -message = CWinPipeIPC::Listen() INVALID_HANDLE
 SystemError Value = 0x000000E7
 
[09/11/2011 21:09:43.296 38244 0] UiInitialConnectThread.cpp:138 - CUiInitialConnect::Run RETURNING...
[09/11/2011 21:09:44.061 41828 0] apps\dsagent\agent\LogEventThread.cpp:57 - CLogEventThread::Run CLogEventError
 Thrown from apps\dsagent\agent\LogEventThread.cpp:113 -message = CLogEventThread::ReadLogEvent() - Shutting down.

[Migration User]

Until yesterday we were on SP4. The INVALID_HANDLE error can be caused by the AV on your server treating named pipes as files. We run Sophos here and I couldn;t find a way to stop it doing that. Disabling on-access scanning solved the issue for about 3-4 weeks and then it came back...

As of yesterday we are now running 6.9 SP5 MR1. Once again the problem seems to have been resolved, the 64bit machines with encryption on in the client are now reporting in and our image jobs complete successfully. But I have been down this road before so i won;t say the upgrade solved the issue until we have been trouble free for 4 weeks or so.

[Migration User]

We are about to make the move to Windows 7 and we are on 6.9 SP4 and this issue is starting to crop up in out environment.

Did the upgrade to SP5 MR1 solve the problem?

[HighTower]

I'm on SP5 MR1 and I have this problem in production but only on 2008 64-bit and 2008 R2 servers.  We do not yet have Win7 deployed to test with (I know, I know...).

[HighTower]

I solved this for good in my environment but turning off "Encrypt session communications with Deployment Server" on the Security tab on the agents.

7 days rock solid without my DAgents dropping offline.

[ianatkin]

Hi all, just seeing this thread now. Historically I've seen few problems with the encryption settings enabled in DS comms. I generally advise that this feature not be enabled.

DS traffic does not what I would normally consider to be sensitive data -basic inventories, software pulls and scripts.

I might have missed something here though. Is there a reason why you guys are using encrypted aclient connections?

Kind Regards,
Ian./