Add MANY URLs to ****, No-SSL, and No-Auth - most efficient way

[JS_2022]

I have a request to add over 200 URL's that require ****, no-SSL-intercept, and No-Authentication. Doing it in the GUI will take a long time. Is there a quicker way to do it form the CLI (or somewhere else? a script?)

[JS_2022]

Not sure why the subject and initial request have **** - that should be white list

[Manfred Proell]

You can make it in CPL language / layer. Or you can add it to a local URL-Filter list and use in your policy. Just two short ideas of maybe many other possible. 

[Klaus Klinge]

Hi Jeff,

If you have a management center, then use a shared object. 
The type should be "Category" because then you can use it both in the web access layer and in all other layers, including a forwarding layer.
After creating, you can add the 200 URLs via "Paste URLs". (Attention, there must be no "https://" at the beginning).

Go to the ruleset and add the shared object:

Click Save - Open VPM Editor.

You can then use the shared object normally in the ruleset: 
Add a new object
Request URL Category
You can find your shared object under Management Center 

Alternative:
Create a CPL_Layer: 

; Beginning of CPL-Layer

define category My_200_URLs

nasa.gov

cnn.com

[...]

playboy.com

end

<Proxy> ; Web Access Layer

    condition=My_200_URLs Allow

<Proxy> ; Authentication Layer

condition=My_200_URLs authenticate(no)

<SSL-Intercept> ; SSL-Intercept Layer

condition=My_200_URLs ssl.forward_proxy(no)

; End of CPL-Layer

Attention: This will not work, when you have an "Force-Deny" in an other Web-Access-Layer. 

For example, if you have an "Any - Any - Force-Deny" rule in another web access layer, check if you can replace the force-deny with a **** deny.

[JS_2022]

Unfortunately, I do not have a mgmt station available.

I found a script for GotoMeeting on Broadcom KB:

https://knowledge.broadcom.com/external/article/166772/problem-accessing-gotoassist-gotomeeting.html

Could I not just modify it like this:

;Many URL RULES
<ssl-intercept>
condition=Many_URLs ssl.forward_proxy(no)

<proxy>
condition=Many_URLs http.method=CONNECT authenticate(no) ALLOW
 
 
;---------------------------------------------------------
;CONDITIONS
 
define condition Many_URLs
url1.com
url2.com
url3.com
url4.com

[.....]
url200.com
end

I removed detect_protocol(no) as it is not required.

[AKH_BC]

I'd NOT recommend using "define condition XXX" with a list of URLs.

If you are using CPL, there are better CPL gestures that are "easier" for SWG to consume and compile and that format likely is NOT the best.

I'd re-visit your choice of CPL gesture on maximizing the parsing and compiling of the CPL when you have a large list, a list of URLs in a Condition is NOT the most effective.

------------------------------
Providing you Premier Partner support via Accredited Training, Professional Services, Consulting Services, Partner support for products with full staff of Broadcom Knights.
Looking for support and want the top versed individuals to assist you? Email [e-mail] and let's start a conversation on how we can make you successful. Award Winning North America Symantec Knight Partner of the Year for 2021.
~~##Groundbreaking Expert Advantage Partner using CA Automic Automation for Automated Policy Rule Implementation with BSG Symantec Products! Visit https://braxtongrant.com/solutions/automation/ or ask me how to get your demo with us on this One Of A Kind Integration!##~~
------------------------------

[Klaus Klinge]

Broadcom allows you to set up a "shared object" as a condition type - which has the advantage that you can use the same **** for web access layers and forwarding layers. 

We have been working with this for years and have had no bad experiences. 

We no longer have to maintain two URL lists and, unlike with a combined object, we can enter a comment for each URL, which is important for us.

I can maintain a shared object in the Management Center - but then I always(!!!) have to work with the VPM editor of the Management Center - the local VPM editor must never be used, because then the shared objects are missing. However, this is not a problem for us. 
In addition, a test has shown that a URL list of type Category can also be maintained in a CPL (layer) - e.g. if you want to assign individual URLs to a specific category. 

It is not clear to me why you advise against this. 

[AKH_BC]

Klaus,

I look at it like this, it's not that Shared Objects are bad, but traditionally a local database (and the way the local database is consumed and parsed by the OS) is a quicker consumption and review of entries vs a Shared Object Category/Category List/IP,etc.

Using MC to manage a Local Database also gives you the ability to write comments (not a dedicated comment field, but you can still (; comment here ) to your heart's content.

How I rate use of elements in policy based on Performance/Benefit to the compiling of policy (Top to bottom as top is best) based on what types of things you can use:

1. Local Database (not applicable for Cloud SWG unfortunately, hopefully in the future?) either managed on MC as a Policy Object or locally hosted (Most Edge SWG at this point can connect to over 5+ different Local DBs).
2. Shared Object on MC for Category/URL/IP
3. Local VPM object (not managed by MC) for Category/URL/IP which can be in VPM or in CPL
4. Combined Object in VPM/CPL that contains individual objects of url.domain= or category= or similar logic
5. Individual rules per each category/domain/URL/etc in a layer

None of the above takes into account Layer Guards/Section Guards, just how the OS will handle these objects, and to reduce the multi-layered "object in object in object" decision tree that adds to policy timing delay.

I agree, MC managed objects means if your MC goes down you can't change policy since the Edge or Cloud SWG has no idea of the contents of those objects outside of what MC gave it.  However you can add policy changes to CPL files if you aren't already using them and they aren't managed by MC temporarily if your MC goes down and you need to make emergency changes.

I also too, recommend MC, for auditing and rollback capability and to manage granular changes and visibility, but I'll always recommend Local DB use (Or MC managed Local Database Object as a Policy Object) when possible (and when a large number of URLs are included) FIRST before a Shared Object.  The benefit of the Local DB over the Shared Object that comes to mind the most?  Edge SWG will sync that Local DB based on what you set, so no "Push this out" Job or execution required, SWG will pull it on it's next check.  Shared Objects will require a "Push this to a device" behavior to install that new policy, vs the Local DB doesn't require a new policy install.
------------------------------
Providing you Premier Partner support via Accredited Training, Professional Services, Consulting Services, Partner support for products with full staff of Broadcom Knights.
Looking for support and want the top versed individuals to assist you? Email [e-mail] and let's start a conversation on how we can make you successful. Award Winning North America Symantec Knight Partner of the Year for 2021.
~~##Groundbreaking Expert Advantage Partner using CA Automic Automation for Automated Policy Rule Implementation with BSG Symantec Products! Visit https://braxtongrant.com/solutions/automation/ or ask me how to get your demo with us on this One Of A Kind Integration!##~~
------------------------------