YouTube URL path in event log

[ITA ISD]

Hi,

I want to customize the event log to show all the path of a YouTube URL.  It shows for YouTube URLs only the domain name of YouTube like below!:

Bluecoat|src=x.x.x.x|srcport=58412|dst=x.x.x.x|dstport=443|username=-|devicetime=[30/05/2021:09:41:58 GMT]|s-action=TUNNELED|sc-status=0|cs-method=unknown|time-taken=2571|sc-bytes=11013|cs-bytes=535|cs-uri-scheme=ssl|cs-host=www.youtube.com|cs-uri-path=/|cs-uri-query=-|cs-uri-extension=-|cs-auth-group=-|rs(Content-Type)=-|cs(User-Agent)=-|cs(Referer)=-|sc-filter-result=OBSERVED|filter-category=Technology/Internet|cs-uri=ssl://www.youtube.com:443/

But for other request it shows the below:

Bluecoat|src=x.x.x.x|srcport=58412|dst=172.67.215.6|dstport=443|username=-|devicetime=[30/05/2021:09:41:58 GMT]|s-action=TUNNELED|sc-status=0|cs-method=unknown|time-taken=2571|sc-bytes=11013|cs-bytes=535|cs-uri-scheme=ssl|cs-host=cdn.foxpush.net|cs-uri-path=/|cs-uri-query=-|cs-uri-extension=-|cs-auth-group=-|rs(Content-Type)=-|cs(User-Agent)=-|cs(Referer)=-|sc-filter-result=OBSERVED|filter-category=Technology/Internet|cs-uri=ssl://cdn.foxpush.net:443/

Another example is:

Bluecoat|src=x.x.x.x|srcport=52782|dst=188.135.7.63|dstport=443|username=-|devicetime=[30/05/2021:10:06:30 GMT]|s-action=TUNNELED|sc-status=0|cs-method=unknown|time-taken=7810053|sc-bytes=809|cs-bytes=1240|cs-uri-scheme=ssl|cs-host=sms.ooredoo.com.om|cs-uri-path=/|cs-uri-query=-|cs-uri-extension=-|cs-auth-group=-|rs(Content-Type)=-|cs(User-Agent)=-|cs(Referer)=-|sc-filter-result=OBSERVED|filter-category=Business/Economy|cs-uri=ssl://sms.ooredoo.com.om:443/

Kindly, guide me how I can show all the path of a YouTube URI, because all requests/events are showing only the domain name not all path URI.

Appreciate your support.

[Paul Riddington]

Hello,

Please confirm if you are intercepting and decrypting SSL traffic.

Regards
Paul Riddington

[ITA ISD]

Hi Paul 

Yes we are.

Regards,,

[Paul Riddington]

Hello,

I am not sure you are because the example logs you provided only show ssl:// rather than https:// and you will only be able to see the full path if the connection has been decrypted i.e. you see it as an https:// request in the logs.

Regards
Paul

[Sakkarin Pichetskul]

Hi,

From access log that you shown, I see only scheme below

cs-uri-scheme=ssl

it's mean you are setting detect protocol or set the proxy setting to SSL in proxy-services.
But it's not mean you are decrypt SSL traffics.

the traffics were decrypt by proxy, in the access log will show scheme to "https".
And for make sure you were setting correct about log format, pls. shared the SSL log format that you used.

------------------------------
Thank you and BR
Sakkarin Pichetskul

System Engineer
nForce Secure Co.,Ltd. [Thailand]
------------------------------

[ITA ISD]

Hi Paul and sakkarin 

Thanks for your responses, please explain to me more about SSL and Https logs format and how I can decrypt the traffic! we are using self signed certificate!
Here is the SSL log format:
date time time-taken c-ip s-action x-rs-certificate-validate-status x-rs-certificate-observed-errors x-cs-ocsp-error x-rs-ocsp-error cs-host s-supplier-name s-supplier-ip s-supplier-country s-supplier-failures x-rs-connection-negotiated-ssl-version x-rs-connection-negotiated-cipher x-rs-connection-negotiated-cipher-size x-rs-certificate-hostname x-rs-certificate-hostname-category x-cs-connection-negotiated-ssl-version x-cs-connection-negotiated-cipher x-cs-connection-negotiated-cipher-size x-cs-certificate-subject s-ip s-sitename x-rs-certificate-hostname-threat-risk

And what I have provided in the request is a customized log format as bellow!

Bluecoat|src=$(c-ip)|srcport=$(c-port)|dst=$(cs-uri-address)|dstport=$(cs-uri-port)|username=$(cs-username)|devicetime=$(gmttime)|s-action=$(s-action)|sc-status=$(sc-status)|cs-method=$(cs-method)|time-taken=$(time-taken)|sc-bytes=$(sc-bytes)|cs-bytes=$(cs-bytes)|cs-uri-scheme=$(cs-uri-scheme)|cs-host=$(cs-host)|cs-uri-path=$(cs-uri-path)|cs-uri-query=$(cs-uri-query)|cs-uri-extension=$(cs-uri-extension)|cs-auth-group=$(cs-auth-group)|rs(Content-Type)=$(rs(Content-Type))|cs(User-Agent)=$(cs(User-Agent))|cs(Referer)=$(cs(Referer))|sc-filter-result=$(sc-filter-result)|filter-category=$(sc-filter-category)|cs-uri=$(cs-uri)

Thanks,

[Paul Riddington]

Hello,

The custom log format looks ok, but you need to implement full SSL interception/decryption and not just detect SSL. It is best if you follow the steps at Configure SSL intercept for an explicit deployment using a self-signed certificate.

Regards
Paul

[ITA ISD]

Hello Paul ,

Thanks for your replay, I noticed that the keyring was not applied in the SSL proxy, I applied it yesterday. Still, the event log looks the same!. Please advise!

Bluecoat|src=x.x.x.x|srcport=54514|dst=172.217.169.238|dstport=443|username=-|devicetime=[07/06/2021:05:49:51 GMT]|s-action=TUNNELED|sc-status=0|cs-method=unknown|time-taken=208041|sc-bytes=1543|cs-bytes=4793|cs-uri-scheme=ssl|cs-host=www.youtube.com|cs-uri-path=/|cs-uri-query=-|cs-uri-extension=-|cs-auth-group=-|rs(Content-Type)=-|cs(User-Agent)=-|cs(Referer)=-|sc-filter-result=OBSERVED|filter-category=Mixed_Content/Potentially_Adult|cs-uri=ssl://www.youtube.com:443/

When reviewing the access logs, the logs are intercepted by SSL only for YouTube app!
 You can compare with other access logs that are not specific.


Thanks,

[Paul Riddington]

Hello,

Please could you provide the policy setup details of how you are enforcing SSL interception, including any bypasses you havein place.

Regards
Paul

[ITA ISD]

Hi Paul,

Thanks, I think I have got the key point from your replay, I checked the SSL policy and rules and I did not find any policy that disable YouTube interception. but we do bypass some users to ANY destination. Therefore, I've created a rule to intercept YouTube. I kept monitoring for one day and this is the result.

Bluecoat|src=x.x.x.x|srcport=50111|dst=216.58.209.142|dstport=443|username=-|devicetime=[09/06/2021:05:03:10 GMT]|s-action=TCP_NC_MISS|sc-status=200|cs-method=POST|time-taken=198|sc-bytes=1218|cs-bytes=30935|cs-uri-scheme=https|cs-host=www.youtube.com|cs-uri-path=/youtubei/v1/log_event|cs-uri-query=?alt=json&key=AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8|cs-uri-extension=-|cs-auth-group=-|rs(Content-Type)=application/json;%20charset=UTF-8|cs(User-Agent)=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.77 Safari/537.36|cs(Referer)=https://www.youtube.com/c/xCaliBR/videos|sc-filter-result=OBSERVED|filter-category=Mixed_Content/Potentially_Adult|cs-uri=https://www.youtube.com/youtubei/v1/log_event?alt=json&key=AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8

Your support is highly appreciated,
Thank you.

[Imani Mayer]

Youtube-dl gets posted around here a lot but it's starting to get excessive. We don't need to have a post about each development change. YouTube-dl still works just not anonymously, and even then people in this thread have reported it works in recent git changes.

lowes employee portal