Blog Viewer

Configuration Archive - A Custom Multi-threaded UIM Probe

By BryanKMorrow posted 07-21-2016 10:33 AM

  

This is a newer version of the probe_configuration_archive that was available on the CA Marketplace. This probe offers the same features, but does so with multi-threading capabilities. This post will hopefully be a living document where people will comment on the current state and also additional features they would like to see implemented. I will attach the latest documentation and probe package to this post, so please subscribe for updates.

 

Probe Features

This multi-threaded probe will archive every probe configuration in your UIM environment and automatically do configuration differentials between the current and previous configurations. This will allow an administrator to track configuration changes in their environment automatically. This probe base is derived from the previous single threaded probe_configuration_archive.

 

Prerequisites

  • UIM Server version 8.0 or greater
  • Java_jre version >= 1.7

 

Probe Configuration

This probe can only be configured using Raw Configure mode.

raw_config_v1.png

ignored_probes: Use a comma separated list of probe configurations you wish to be excluded from collection

archive_path: File system location where you wish to store the configuration files

threads: The number of threads used while doing probe configuration gathering

 

RUNNING THE ARCHIVE

The current version is driven from the rebuild_configuration_database callback, please run this whenever you want to archive your configurations. Future version will re-implement an interval based archive.

probe_utility.png

 

TECHNICAL INFORMATION

The probe uses an internal JDBC capable database (http://www.h2database.com) to store the probe configurations and also the configuration change data. This allows the data to still be accessible for custom dashboards and reports, but also avoids using the UIM database as a datastore. I chose to stick with the raw_configure option as opposed to the newer Probe Framework SDK because of the multi-threading that was implemented. The multi-threading should speed up the configuration gathering quite significantly compared to the older probe_configuration_archive.

CONFIGURATION_ARCHIVE_CURRENT TABLE

This table contains a record of each probe configuration. It stores the origin, hub, robot, probe, probe version, filesystem path to configuration file, complete configuration file in a column, a record id and archive date.

CONFIGURATION_CHANGES TABLE

This table contains a record of each probe configuration change. It stores the hub, robot, probe, configuration section, configuration key, previous and current configuration value, previous and current archive date and previous and current record ids.

HOW IT WORKS

The probe uses the native callbacks provided in many probes to get the probe configurations.

  1. It uses the gethubs callback on the hub probe to get a list of available hubs
  2. Loops through each of those hubs to get a list of robots using the hub’s getrobots callback
  3. (Multi-threaded) Grabs a list of probes and information on each probe from the controller via the probe_list callback
  4. (Multi-threaded) Saves the configuration file from the robot by using the text_file_get callback
  5. (Multi-threaded) Inserts information about the configuration into the database
  6. Compares the current and previous configuration to determine if there were any changes
  7. If there are changes, writes those changes to the database

 

JDBC CONNECTIVITY

In order to query the data from the UMP you will need to copy the h2 database driver jar file into the <Nimsoft_installation_directory>\probes\service\wasp\libs directory. The driver can be found Here. You will then need to create the data source in the Dashboard Designer.

database_configuration.png

  • Name: Logical name for the connection
  • Driver Class: org.h2.Driver
  • JDBC URL: jdbc:h2:tcp://<IPADDRESS_OF_PROBE_CONFIGURATION_ROBOT/./configuration_archive
  • User: uim
  • Password: uim

 

KNOWN BUGS

  • Ignore List function is currently not working
  • Sometimes when re-deploying you may need to deactivate->reactivate the probe for it to start properly

TODO LIST

  • Enable file and table retention
  • Add callbacks for adhoc differential between configurations based on record/date
  • Add callback option to replace current probe configuration with archived configuration

 

Again, please feel free to comment directly on this post with any feedback or suggestions for the probe. I plan to continue development on this for quite some time.

 

Thanks,

 

Bryan

16 comments
29 views