A common issue for Inventory Solution is for the Inventory Task to crash or hang in one way or another. This article discusses the common sources for the problem, including how to configure Inventory Solution to avoid them, or how to troubleshoot the issue if the standard fixes do not resolve the issue. Also included is the methodology of how to pinpoint where the issue is coming from, including how to generally fix the issues we have seen.

Introduction

Often the only indication of a problem with the Inventory Tasks is an inconsistency in Reports. The Tasks themselves do report execution events, but if the corresponding Task reports are not being viewed, only symptoms such as incomplete or old inventory for individual systems, or systems missing from a report they should appear in reveals this potential issue. Crashes will produce errors both in the Altiris Execution event and at time in the Windows Event log, but hanging issues are only reported that the Altiris Agent killed the process after 6 hours (default behavior).

General Troubleshooting

The following procedure walks through identifying where the problem is located. Once a specific agent or area is identified, the subsequent section labeled 'Agent Troubleshooting' will be referenced to continue the process. Use this to identify what process is failing.

1. Once you've identified a system with the problem (whether one of many experiencing the issue or a single instance) log onto the system for analysis.
2. Open the Agent UI by double-clicking on the Altiris system tray icon, or by running C:\Program Files\Altiris\Altiris Agent\AeXAgentActivate.exe.
3. Double-click on the Package labeled Inventory Agent Package.
4. Click on the 'Run History' tab. This will list the statuses of when the Inventory Task shown in the top dropdown. Depending on the status, continue by applicable step listed below:
   • Task Terminated - Move on to step 5.
   • Run Failed - Jump to step 10
   • Run Completed Asynchronously - Jump to step 12
5. When the status indicates the task was terminated, this means the Altiris Agent ended the task because it had reached the max length for execution. This was done to avoid hung executions from remaining resident in memory. It is assumed if 6 hours passes that the execution is not behaving correctly. This can be changed. The source of the hang must be determined. The parent process is AeXInvSoln.exe, but The following processes have been known to hang or still be running after 6 hours:
   • AeXAuditPls.exe
   • AeXMachInv.exe
   • AeXSNPlus.exe
6. To check for a hang, run an Inventory Task with Task Manager running. After 3 hours or so go back and check to see which of the Inventory Agents is hanging. You'll see AeXInvSoln.exe, but there should be another Inventory Agent listed, as in this example:
   

   Click to view.

7. For AeXAuditPls, normally this isn't a true 'hang'. The inclusion of mapped network drives or the 'Slow Mode Delay' feature can cause the scan to literally take over 6 hours. See the section 'Audit Scan' in the subsequent segment for details on how to resolve this.
8. For AeXMachInv.exe, this is typically a hang situation where a loop is occurring with a WMI query or the process has hung. See the section 'Machine Inventory' in the Agent Troubleshooting section for details on how to resolve this issue.
9. For AeXSNPlus.exe, we do have a test build for a similar WMI problem associated with AeXMachInv.exe. Please contact Altiris Support for details. While this problem isn't widely seen yet, I wanted to make mention of it just in case.
10. Run failed means something in the Inventory Task crashed or exited with a failed exit code. Check the Event Log, and if an error resides search the Altiris Knowledgebase for a solution. Otherwise use the section below entitled Revealing the Execution. If it is an application error, refer to step 5 of the Machine Inventory section.
11. Using the details from where the execution fails, search the Altiris Knowledgebase. Most common errors are documented therein and should aide in resolving the issue.
12. The message 'Run Completed Asynchronously' does not necessarily mean the execution was successful. This means that the Altiris Agent was restarted before a return status came from the running execution. This may indicate a hang. Start at step 6 to see if a hang is occurring. If that isn't the case, you can use the 'Revealing the Execution' section to see how the execution launches.
13. This completes the steps for General Troubleshooting. If this doesn't cover your scenario, check the Altiris Knowledgebase or call Altiris Support for assistance.

Agent Troubleshooting

This section covers individual sections to extend steps included in the General Troubleshooting section above. The changes suggested here resolve common issues to the hanging or crashing problems in the Inventory Solution Agents.

Audit Scan

If the AeXAuditPls.exe is taking a long time to complete, follow these steps to adjust the scan so it completes in good time in your environment:

1. On the Notification Server browse in the file system to install_path\Program Files\Altiris\Notification Server\NSCap\Bin\Win32\X86\
2. Execute the file AeXAPEdit.exe.
3. Go to File > Open and browse to \NSCap\Bin\Win32\X86\Inventory Solution and open the file auditpls.ini.
4. Click on the Advanced tab.
5. Under the section 'Slow mode Delay' is there a value posted? Recommended values are between 5 and 20. If it's higher, try lowering it, starting at 5 and move up if performance is still an issue.
6. Also look under the Directories tab. Add drive exclusions by letter for network drives that may have large amounts of files within. The format is:
   • H:\

Machine Inventory

The following steps should be followed to resolve a hanging issue with AeXMachInv.exe. These steps will cover most of the common causes of this Inventory Agent hanging.

1. On the Notification Server browse in the file system to install_path\Program Files\Altiris\Notification Server\NSCap\Bin\Win32\X86\Inventory Solution
2. Open the following files and make the same change detailed in step 3:
   • AeXInvSolnAdm1.ini
   • AeXInvSolnAdm2.ini
   • AeXInvSolnAdm3.ini
   • AeXInvSolnUsr1.ini
3. Add to the AeXMachInv.exe command-line the following command-line switch:

   AeXMachInv.exe /norbin
   
   

   Note: If other switches are present, simply add the /norbin switch leaving the other arguments present

4. Test the above change to see if it resolves the issue. If it does not, continue on with step 5. The above change instructs MachInv to skip any recycling bins on the system. The bins often have corrupt files therein, which causes MachInv to hang.
5. Open the same files as previously indicated and add the following command-line argument:

   AeXMachInv.exe /wmisync
   
   

   

   Click to view.

6. Now test the execution to see if this resolves the hang. The /wmisync switch changes WMI queries being run by MachInv from asynchronous to synchronous. We've seen errors where an asynchronous query behaves poorly and causes either an application crash or a loop condition.

Revealing the Execution

To reveal how Inventory is running, use the following steps. This will show all execution and allow any errors to be displayed onscreen.

1. First we need to remove all the quiet or silent switches from the command lines. Browse in the file system to install_path\Program Files\Altiris\Notification Server\NSCap\Bin\Win32\X86\Inventory Solution.
2. Clone the file AeXInvSolnAdm2.ini and rename the clone to AeXInvSolnAdm0.ini.
3. Remove the /hidden switch from any command-line listed (default is the AeXAuditPls.exe and AeXInvCollector.exe) and save the file.
4. In the Altiris Console, browse to View > Solutions > Inventory Solution > Tasks > Windows > Recreate Full Inventory (or if another task is the only one exhibited the problem, select that one) and clone it (right-click > Clone).
5. Click the link 'Go to Program'.
6. Choose the Program 'Full definition file with CleanBeforeRun option' from the top dropdown and copy the command-line out.
7. Click the 'New' button next to the Program dropdown.
8. Type a Name for the Program such as 'Revealed Inventory Execution' and paste the command-line in that you copied from step 6 into the command line field.
9. Edit the command-line to point to the new INI and remove the /hidden and /s switches, for example:

   AeXInvSoln.exe /cleanbeforerun AeXInvSolnAdm0.ini
   
   

10. Click Apply to save the changes.
11. Create a collection based of one or two systems to test with.
12. Assign the above collection to the cloned Inventory Task.
13. In the Program details (found as a tab within the configuration of the package), set the Program can run: option to "Only when a user is logged on."
14. Check the option User Input Required and select 'Normal' for the starting window.
15. Click Apply.
    

    Click to view.

16. On the test system, update the configuration by going into the Agent UI (double-click on the tray icon, or if the icon is hidden, execute the file C:\Program Files\Altiris\Altiris Agent\AeXAgentActivate.exe) and click the icon to the left of the #Help icon at the top of the UI, and then click the option Update Configuration. To confirm the new changes were received, both the Last Configuration Requested and Last Configuration Changed dates should be current.
17. Back to the main agent UI, select the Inventory Agent Package and click the link under Application Tasks to execute.
18. The program will execute and you will see the execution.

Caution! You want to do this carefully as you do not want to show the execution to all your end-users. This is why we create new objects for the entire execution, including a test collection.

Conclusion

A large part of the above steps is figuring out what is occurring by observing the execution, and then taking action per the results. Since Inventory Solution Agents are not integrated into the Altiris Agent, we have the ability to manipulate the execution easily via the INI files, command line, and execution environment presented by the Program. Hopefully this covers most instances of the hanging or crashing issues.