Automatic distribution with manual pre-staging does not work

[Tomasz Woźniak]

Hello guys,

Here is the scenario. By default we distribute updates to all package servers. Recently we introduced Office ProPlus. To support patching Office365 in 27 countries we enabled several languages in the "Import Patch Data for Windows" task. Additionally the following Windows 10 versions: 2015 LTSB, 2016 LTSB, 1607,1709,1809 and Server 2016 are currently supported by Microsoft.In result our Package servers download about 20-30 GBytes of data every month. We are simply running out of free disk space very quickly.

As one of the remediation actions I changed the "Package Distribution" settings in "Windows Patch Remediation Settings" from "All package servers" to "Automatic distribution with manual pre-staging". I changed the setting to "Delete packages after 1 day". Additionally I changed the following settings in "Package Service Settings" page:

- Remove automatic site assignments if they are unused for 1 week

- Delete package files if they are unused for 1 day.

The problem is after 1 months since change,  the already downloaded updates have not been deleted from package servers. There are tons of old updates cached on PS no longer relevant for XP, Server 2003 etc, wchich I am sure are not requested by clients in given site. We have hierarchy and I made the change on parent only to start with.

What am I missing ? Any ideas ?

It is becoming pretty urgent.

Thanks

Tomasz

[jvall_cae]

Hi Tomasz,

You will need to disable old policies (allow time for config updates) and then delete the policies from the console. The actual removal takes place when the Integrity Check runs. Look under System Jobs and Tasks -> Software -> Patch Management -> Check Software Update Package Integrity, and make sure you have a schedule setup to run daily.

 

jv

[Tyler Russell]

Tomasz/jv

 

This is an extremely relevant topic for me as I'm running into the exact same issue. 

 

@jv - What is the best way to determine old policies that are able to be disabled? Bulletins that have reached 100% compliance? Bulletins that have been superseded? As an FYI, we're using the Patch Automation Tool that was built by Ludovic (not sure if that would play a role in this process).

 

Appreciate your help.

[amandaw33]

@Tyler,

I run the superseded report monthly and move any policy that is fully superseded to a "disabled updates" folder in the policy list on the left.  If you've never done this, you'll want to make sure when you run the report you select from date from like year 2000 or something to catch all old updates.

Once they're all in there, I click on the disabled updates folder and then in the right pane I can select all policies and right click and disable them all.

24+ hours later, I then delete those disabled policies.  I'm not sure if the 24 hour wait is still necessary, but at one point support had told me it was good practice.

In patch remediation center, then, once no policies are assigned to the superseded bulletin but it still shows downloaded YES, I right click on the bulletin and select disable.  At this point, at next clean up interval, the server will clean up those disabled bulletin downloads. 

I am not at work today to give you exact job name, but you can also force the clean up once the bulletins are disabled by going to jobs/tasks, software, patch and there is a job there you can run that cleans up the old bulletins on demand.  Let me know if you can't find it.

I don't delete policies that are 100% compliant in case somehow some old machine comes back online and needs the updates.

[Tyler Russell]

Hi Sally, 

 

Thank you so much for this description. 

I've read the HOWTO on what to do with old policies but wasn't sure what the "Real World" best practice was. So it's great to hear how an actual company and Symantec ITMS Admin is handling this. 

It cleared things up tremendously. 

Much appreciated!

Cheers!

[Tomasz Woźniak]

Duplicated comment by error.

[AlexD]

Hello,

* "Automatic distribution with manual pre-staging" setting when enabled takes effect only for new packages. Old package retain original setting.

* What sites should receive package is determined by policy/target, not actual demand by client. So when policy is created server gets all targetted resources and finds all affected Sites and then all packages servers within those sites are getting packages.

* "Delete package files if they are unused for" setting controls when to delete package when it no longer provided by SMP. It has nothing to do with actual demand.

 

If you do not have Windows XP/Server 2003 we recommend to exclude updates when creating Software Update Policy to save disk space.

Recent changes in Windows update now require more space. Consider adding more disk space for package servers responsible to distribute windows updates.

[Tomasz Woźniak]

Hi Jv,

Thank you for your reply.

I am afraid that your answer is not relevant to my scenario. I have no issues in deleting disabled policies and disabled bulletins. The "Check Software Update Package Integrity" works just fine daily.

The question is about "Package Servers automatically with prestaging" settings.

 

Just to cast more light. The parent server manages exactly 3 computers: NS itself and 2 Site Servers.

All 3 computers are Windows 2008 Server R2. 

My questions is why the updates packages are still cached on Package Server despite. My understanding is with these settings the Package servers should have Windows Server 2008 applicable update and only requested by those computers. For some reason there are still all updates cached on those Package servers.

OK while writing the response to this comment, I actually found the kb that answers my question.

https://support.symantec.com/en_US/article.HOWTO38239.html

At the very bottom it reads:

"This will manage each package moving forward on all newly created Software Update Policies"

This simply explains why the existing packages are nor deleted even they are not applicable to the managed computers.

I believe the kb on the overall is very useful for anyone wanting to clean up their their Patch Management estate.

 

Here is the additional information I received from Symantec support. In my case another reason why the packages were not removed is the package server was in the same site as SMP server.

 

Automatic Package Server Assignment

Overview

Automatic package server assignment aims to assign packages to sites as required in order to distribute the package to all parties which are assigned the package to download. It also adds support to have these assignments removed automatically after they have not been used for a specified number of days.

Automatic Package Server Assignment in the UI

Automatic package server assignment is an option in the package servers tab of a package. The user can manually select sites which contain the package servers to assign the package to but an additional set of sites determined from automatic package server assignment will be added to it. These two sets, manually assigned and automatically assigned are treated separately to allow the un-setting of manual assignments to sites. Automatic assignments to sites are only ever unset by the automatic un-assignment schedule.

How automatic assignments are determined

The current set of automatic assignments is the collection of sites which contain computers which are set to receive the package. This is determined in a series of steps. First the set of enabled software tasks which reference the package is obtained. Next all resource targets which are referenced by any of the tasks in that set are gathered. These resource targets contain the computers which are set to receive the package. This set of computers then has their IP addresses retrieved and the set of sites which are the closest encompassing sites for any of the IP addresses in the collection is created. This set of sites contains the sites which are to be automatically assigned. This process takes place in SWDSupport.SetPackageServerAssignment.

Special cases to pay attention :

• Specifically if any of the computers are in a resource target assigned to a site or site server, then that site or the site which contains that site server is used instead of the site which contains the computers IP address. Resource target assignment to an internet site does not override the IP address behaviour of a computer when it is not in the internet site.
• Every package is always assigned to any sites which contain the SMP server.
• Every package is always assigned to Default Internet Site.
• In case no sites are defined (except Default Internet Site) every package is always assigned to ALL package servers

How automatic assignments are used

The database stored procedure spSetPackageServers takes care of updating the database to match the package server assignment. It receives both the set of automatic assignments chosen and the manually assigned sites. Sites which have previously been automatically assigned are added to the automatic assignments group passed to the stored procedure. This allows the addition of 'manual' automatic assignments.
The stored procedure first removes all manually assigned sites from the list of automatically assigned sites. This gives manually assigned sites precedence. Then all sites which have been previously been automatically assigned are removed from the list to be automatically assigned. This is done by checking the package site activity log which has an entry added to it when assignment first occurs, and when subsequent requests come in from agents in a site requesting a package. All package site assignments which were manually assigned for the current package are then removed. The new manual assignments are then added back in, except ones which already have a corresponding pre-existing automatic assignment. The new automatic assignments are then added in unless there is already a manual assignment.
All new automatic assignments then have initial entries added to the package site activity log where there are currently no entries at all. Once that is completed the process of determining package servers which receive packages is the same as for the manually assignment to sites case. The distinct set of package servers associated with all assigned sites is obtained and all of those package servers are assigned as if individual assignment was selected.

Automatic Assignment as Required

In addition the standard automatic assignment process there is automatic assignment as required. This is triggered when a request comes in to GetPackageInfo which receives no download locations. If there are no download locations and the package has automatic assignment set then an automatic assignment between the client's closest sites and the package is added. This assignment is forced, that is it is added even if there are entries in the package site activity log. This allows automatic package assignment after automatic un-assignment has occurred. Once added to SWDPackageSite the standard assignment process is followed. This takes place within spForceAutoAssignPackageSite.

Automatic Un-assignment

Automatic un-assignment is the process of removing package site assignments when they have not been used for a period of time. This is performed on a schedule which runs daily. The package site activity log is checked for each package assigned to a site and for each site where there has been no activity in greater then N days, automatic assignments to the site are removed. The activity log is unchanged by this, which stops automatic assignment from automatically re-assigning the package next time assignments are updated. The assignment can still be recreated, but only by the assignment as required which bypasses the activity log check.