Release Automation

Nolio RA Collect Logs Scripts

  • 1.  Nolio RA Collect Logs Scripts

    Broadcom Employee
    Posted 02-10-2020 11:27 PM
      |   view attached

    Please note that these scripts are not supported by our development team, support team, or Broadcom. They are community supported. They are being provided as a courtesy to anyone they might help. Enhancements, ideas, fixes, feedback, etc.. via the Community is encouraged.  The support team will not fix problems you have with these scripts. If anyone is interested in using and sharing updates to these scripts (and more - see dev notes) then post a comment. We can look into setting up a git repository where people can submit updates, issues, get latest, etc..

    The topics of this article include:

    • Features
    • Requirements
    • Instructions
    • Script Descriptions
    • Notes (updates needed inside of the scripts)
    • Mac user notes
    • Future Ideas
    • Development environment

    Request:

    If anyone has something (probably powershell, maybe ansible or chef) that already collects logs from NAC|NES|NAG on Windows servers then please share. Otherwise, look for one being released hopefully soon (though later than I was hoping  - sorry).

    Features:

    1. Tar the logs directory and nimi_config.xml (if file exists) on a user defined group of servers.
    2. Copy the file from the remote server to your local workstation - in the folder where the script is run from.

    Requirements:

    - There are a few others described below, but most notably is that these scripts bash/zsh scripts. So a bash/zsh is the bare minimum requirement. I am encouraged by script compatibility test I've run (not with these specific scripts) on Windows and Linux (see dev notes below) that lead me to believe that these scripts would likely also work on Windows 10 bash shell feature.

    Instructions:

    1. Download and extract the archive file with the scripts.
    2. Determine which script file is appropriate for your situation. See "Script Descriptions" below.
    3. Make the necessary changes to the script. See "Notes" below.
    4. Make sure the script is executable: chmod 700 <script name from step2>
    5. Run the script: ./<script name from step2>
    6. Find the remoteServer's tar file in your workstation's local directory: nolio_<remoteServerName>_<date>.tar

    Script Descriptions:

    The 4 scripts that collect log files can be categorized into two groups.

    Group 1: Connects to the remote server as the user that owns the nolio installation folder.

    Group2: Connects to the remote server as a user that does NOT own the nolio installation folder, but can "sudo su - <nolio_username>" without being prompted for a password (via sudo configuration NOPASSWD:).

    Group3: Each group has a pair of scripts. The difference between them is whether the connection requires an ssh password or not to initiate the connection with the remote server. The scripts were named to reflect expectations and requirements.

    Group 1 scripts:

    collectLogs.no_sshpass.sh assumes:

      1. You are able to connect (via ssh), to the user defined list of remote servers, as the user that DOES own the nolio installation folder on the remote machine.
      2. While connecting as $nolio_user you are NOT prompted for a password (using ssh keys with no passphrase).

    collectLogs.sshpass.sh assumes:

      1. You are able to connect (via ssh), to the user defined list of remote servers, as the user that does NOT own the nolio installation folder on the remote machine.
      2. While connecting as $nolio_user you ARE prompted for a password (original release does not support ssh key passphrases yet).

    Group 2 scripts:

    collectLogs.sudo-nopass.no_sshpass.sh:

      1. You are able to connect (via ssh), to the user defined list of remote servers, as the user that DOES own the nolio installation folder on the remote machine.
      2. While connecting as $nolio_user you are NOT prompted for a password (using ssh keys with no passphrase).

    collectLogs.sudo-nopass.sshpass.sh:

      1. You are able to connect (via ssh), to the user defined list of remote servers, as the user that does NOT own the nolio installation folder on the remote machine.
      2. While connecting as $nolio_user you ARE prompted for a password (original release does not support ssh key passphrases yet).



    Notes:

    Notes (all scripts):

    Before any of the collectLogs script(s) can be used there are a few variables that need to be updated.

    cara_home: This variable needs to be set to the absolute folder path/location of the cara component you want to collect the logs from.

    carauser: This variable needs to be set to the username that owns the cara_home folder.

    done <<EOF

    <host1>

    <host2>

    EOF: this section needs to be updated with the list of hosts that you want to run the script against.

    Notes (for collectLogs.sshpass.sudo-nopass.sh  &  collectLogs.no_sshpass.sudo-nopass.sh):

    loginuser: This variable needs to be set to the username that your workstation uses to connect to the cara server.

    MacOS Note:

    If your running these scripts on macos then you will likely need to comment out the statement towards the top of the script: set -eu -o pipefail

    The reason why you might need to comment out that statement is because it will cause the script to exit if there are any errors at all. So, for example, if you're running the script against a NAC only server (that doesn't have conf/nimi_config.xml) then the script would error out while it tries running the tar command - and it would not scp the file to your workstation.

    Future Ideas:

    Windows script to compatible set of scripts.

    It shouldn't take too much to combine into one script if there is an interest.

    We could look into putting this into a git repository if there is an interest.

    Variations of scripts based on different connection use cases.

    Ansible Roles to collect logs (install nac, nes, nag?)

    Development environment/notes:

    My main system is a macOS High Sierra 10.13.6. With it I predominantly run zsh. I've been doing a lot of scripting lately. During this time I've found rare occasions where a script line/statement needed to be unique based on the shell being zsh and/or bash. The scripts in the originally posted version does not contain any of these unique shell specific statements. They should run on both. If it's a problem in bash then probably a problem in zsh. With that being said I recommend zsh :)

    My workstation has the following shell versions:

    - zsh 5.3 (x86_64-apple-darwin17.0)

    - GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin17)

    What I have found more frequently are command statements that differ across platforms. For example, date -v isn't on

    Most of the scripts I write are now circulating and being semi-used/tested on the following platforms/servers/workstations:

    • CentOS Linux release 7.6.1810 (Core)
      • zsh 5.0.2 (x86_64-redhat-linux-gnu)
      • GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu)
    • Windows 10: version 1809
      • cygwin: zsh 5.5.1 (x86_64-unknown-cygwin)
      • cygwin: GNU bash, version 4.4.12(3)-release (x86_64-unknown-cygwin)

    These scripts have not been tested/used on anything other than my main workstation.


    A BIG thanks to Yuri for the original script that got these variations going! And a BIG thanks to ALL OF YOU for using Nolio, CA Release Automation!

    I have some other scripts. If there is an interest then I can look into share them sooner than later. Some features of some other scripts:

    • Extract the files.
    • Summarizing start/end date & timestamps collected in the specific groups of files - per server.
    • Scripts to start file reviews using the less pager utility in zsh. It starts less passing a regular expression pattern focused on a specific set of messages (as indicated by the script name). For example: Artifact distribution related messages, Action File distribution related messages, etc..
    • Awk scripts for highlighting log4j log levels differently.

    NOTE:
    Some (very few) of these unpublished/unshared scripts described above may have statements that are specific to the shell. And some require some rust based tools (fd, fzf, bat) to function properly - though hopefully the assertion tests (as seen in these scripts) will be up to par which makes it very clear that those tools are not found and exits the script.


    Cheers,
    Gregg

    Attachment(s)