Symantec Access Management

 View Only

Tech Note : Howto place SSO policy changes under revision control using git 

May 01, 2017 11:53 PM

1. Introduction 

 

Here is what we are aiming to achieve.  We would like to keep a record of the changes made to a policy store under revision control, and to be able to see when policy changes have occurred and view the differences between revisions. 

 

The above picture shows the revision history diaog and in the background (in blue/red/black) a diff comparison of the current revision with the policy store from 6 days ago.

 

Note: But first a warning: this software is a CA Community development initiative, it is not an official CA product, and this is a first release of the new ssobackup functionality.  Having said that the underlying mechanisms (git & ssh in particular) are fairly solid, so we expect it will be fairly reliable, but there will still be some teething issues and possibly some structural changes.  We are releasing it in the hope that people will try it, and give us some feedback and thoughts on how the process should evolve - Cheers Mark. 

 

This document is about implementing the ssobackup process, If you are after an overview and demo showing viewing change history then this shorter article is more appropriate :  Tech Note : Storing SSO policy changes in Revision Control - viewing changes 

 

The tools for storing text files and source code under revision control have a long history are very robust.  Also the SSO XPSExport policy store export format is a close, but not quite perfect, fit for storage under revision control.   What we do in this article (and program) is :

a) Tinker with the SSO xpsexport format a little to make it compatible with a revision control system.

b) Write a batch job that commits differences in the modified policy changes. 

c) Modify the SMPolicyReader so it can list the git revision history and compare revision versions. 

 

 

Process Overview :   Here is what we have implemented :

 

 

The concept is to be able to store a group of related policy stores, eg, DEV, QA, PROD etc in a group in the same PolicyRootEnv directory or repository.  

 

The programs used are : 

ssobackup: we have a small program ssobackup (part of the SMPolicyReader package) that runs regularly (once per hour) via a scheduled task/cron job on a policy server in each of the environments.  When it runs it does an XPSExport and manipulates the output to make it more "revision control" friendly, and checks in any changes that are found.  

 

SMPolicyReader:  The policy reader has the ability to fetch the current policy stores stored in Git and also to view and read the revision history.  It can display a list of changes for the policy store as a whole or list of changes for a selected app/domain.   The user has some control over what amount of history is shown, and can then view or compare revisions. 

 

The model we are using is revision control stored in a GIT server with access to the Git server via a SSH client.    We are conscious that others will have slightly different requirements, so in our design we've tried to isolate the interface to the revision control software to to allow other implementations to be easily developed (For example we expect to have a GIT server via HTTPS access (with UN/PW) to github without too much trouble in the near future). 

 

The ssobackup process and SMPolicyReader are designed and run on Windows and Linux platforms (I tend to develop on Linux, and then run the testing on Windows) - so it should run on both those platforms. 

 

In the below steps we setup and install the ssobackup and policy reader process on one policy server.  The process can be repeated on each different environment and on any developer or admin desktop where access is needed.  The install steps, each a section below are : 

 

  • Prerequisites 
  • Installing Git & SSH 
  • Setup of SSH to use public/private key pairs
  • Install SMPolicyReader on dev policy server. 
  • Setup initial repository and dev policy env
  • Setup ssobackup program to run on policy server.
  • View policy change history on dev policy server. 

 

2. Prerequisites

Requirements for this setup are :

Git : The revision control software :   https://git-scm.com/ 

ssh:   Secure Shell :  https://www.openssh.com/ 

SSO SMPolicy Reader : Siteminder Policy Reader   via the CA Communitry page 

 

For Linux/unix Git/ssh generally come as a package to install with the operating system.  For windows, there are a number of different packages to choose and to keep it simple I walk though the Windows installs of git and ssh that I have used.  

 

The SMPolicyReader, which can already do a compare/diff of XPS exported policy stores, now has a subdirectory/module called : ssobackup.  

 

 

3. Installing Git & SSH 

We need to install Git and ssh on a policy server in each environment, and on any desktop that intends to view the policy store.

 

3.1 On Windows

For the windows program installs I have left it to use the default install locations. 

 

Git for windows

Here is where I downloaded my GIT client for windows :

"Git for Windows"   https://git-for-windows.github.io/ 
https://git-scm.com/download/win 

 

This guide was also useful:  

http://guides.beanstalkapp.com/version-control/git-on-windows.html

 

SSH for windows

There are a number of ports/implementations of openSSH for windows, putty being the most common one.  But to keep it simple, the one I used was - although on install you only need the client part (ie not the sshd server component).

https://www.mls-software.com/opensshd.html

 

 

3.2 On Linux

You probably already have them installed, otherwise fire up the package manager and select them - then get a cup of coffee wait for your windows colleagues to catch up, and then move to the next section.

 

4. Setup of SSH to use public/private key pairs 

To communicate with the Git server, via SSH we need to generate an RSA key pair.  Note: When prompted for a password to store the key pair please just hit enter so they are stored without a password.   This means the user has access to the public/private key pair whenever they are logged on or from a process that is running as a scheduled job by the user.    The process is identical on Linux and Windows.  

 

"c:\Program Files\OpenSSH\bin\ssh-keygen.exe" -t rsa

 

 

You will need the contents of the id_rsa.pub key, which is your public key, in the next section.  Basically you need to place the public key (id_rsa.pub) in the Git server so that when you connect it knows you are a trusted connection.  The rsa public key looks like this: 

 

Note: the other file id_rsa is your private key and you should not share this file with anyone - its a bit like giving your password away.  It is not that unusual for an unknowing user sends both the public and the private key files to a third party. 

 

 

5. Create a Repository on your Git Server

We need a Git repository to store the date, that can either be one on your own Git Server, or the Git server can be one provided by an external supplier, we show both below, and for the external supplier we've used github.com. 

 

5.1 Your own Git Server & Repository

Hopefully you have someone that has already setup your company Git server, and you can just ask them to add you to the project.  I did set one up and this guide was what I used.  For the remote access I've used SSH access (rather than https - latter on we will revisit the https option). 

 

Generating Git Server & creating a Git Repository 

The guide for setting up a server and repository on Unix is: 

https://git-scm.com/book/en/v2/Git-on-the-Server-Setting-Up-the-Server 

 

For me, after setting up the git server,  this was how I created the repository: 

 

root@odoma04-vm2:~# su - git
git@odoma04-vm2:~$ cd /opt/git
git@odoma04-vm2:/opt/git$ ls
project.git
git@odoma04-vm2:/opt/git$ mkdir PolicyEnvRoot.git
git@odoma04-vm2:/opt/git$ cd PolicyEnvRoot.git/
git@odoma04-vm2:/opt/git/PolicyEnvRoot.git$ git init --bare
Initialized empty Git repository in /opt/git/PolicyEnvRoot.git/
git@odoma04-vm2:/opt/git/PolicyEnvRoot.git$ 

Note: after creating the repository, I reset the git user so it is not interactive as per the instructions link above.: 

 

Adding your public key to the git server.   
For Linux you just need to append your public key to the end of the ~git/.ssh/authorized_keys file : 
    cat >> ~git/.ssh/authorized_keys
But presumably you can send your public key to someone who will do that for you: 

 

 

 

5.2 Using Github Repository

An alternative to rolling you own git server, is Github.   Github.com is a cloud based provider for storing git repositories, they makes setting up and using Git repositories fairly easy.  By default the (free) service give public access to your repositories, presumably if that is our choice you will opt for the paid for service.   

 

Creating a Git Repository 

The githib.com screen for generating a new repository is : 

 

 

Adding your public key to the github server.   

The github screen for adding your RSA public key to your profile.  you click on your picture, and select "settings' and then select "SSH and GPG keys". 

 

5.3 Test Access to you Github Repository

In my case, with my own private git server.  I've added an entry into the /etc/hosts file to point the name gitserver to the IP address of the host I installed a git server on. 

 

To access the git repository, we need to  run :   ssh git@gitserver

On first access to a new host, it will ask if you trust that remote host, that protects against future man in the middle attacks

 

In the above example it then prompts for a password, that was because I had not setup the ssh keypair as yet.

 

I then set them up, and added the public key as trusted and the 2nd access below, then connects and authenticates, but then logs me out, that is because the git user is set on the server to not allow interactive sessions - but that  is ok, we only wanted  to verify the authentication. 

 

We have now completed testing access to our git server. 

 

6. Install SMPolicyReader on dev policy server 

The SMPolicyReader can be downloaded from the CA communities site : Siteminder Policy Reader  

It is a simple .zip file eg : SMPolicyReader-4_0-ALPHA-426_bin.zip   It contains both the SMPolicyReader and the ssobackup programs. 

 

I tend to install the program into a c:\apps directory  - I also usually to keep the version # name, but you may wish to rename it to a non versioned name - as we will be latter adding a scheduled task pointing into this directory. 

 

To run the SMPolicyReader usually via run.bat/run.sh gives the best results.

 

 Note1: If you directly click on the .jar file, often you end up running via the 32bit java, even if you have 64bit java installed.  64bit gives more space, and so it is better to click on the run.bat / run.sh file to start it.


 Note2: The runOracle.bat is only needed if you are trying to connect to an Oracle ODBC policy store to read the contents as there were some extra .jar files that we could not dynamically add to the path.  However, here we are only accessing via the XPSExport, it does not hurt, but you do not need to run that way.

 

7. Setup initial repository and dev policy env

We want to create a PolicyEnvRoot, which is a directory where we will have subdirectory for each different env realted to this group of policy stores.  In the below example we will only create one env a DEV environment, but the process is similar when creating multiple ones.   We then will push the directory structure as our initial commit into the git reprository. 

Note: We expect to create cmd line version of these steps in future to assist with install on remote non gui servers.  

 

7.1  Create PolicyEnvRoot directory

First we create the PolicyEnvRoot directory.  It is where we will store the collection of policy environments.   It is also the root directory that is stored in the Git repository.  We create this from a menu in the SMPolicyReader and after prompting for some parameters it creates the directory and also pushes the initial setup to the repository.  The screens are :  ( there is also a cmd line option as well - will discuss in latter post).

 

 

First select the menu item Options/New Env Root.

 

On the policy server we want the "Create New" option, since we don't have a PolicyEnvRoot as yet.  But if we already have a PolicyEnvRoot created, then we can use clone to connect to it, rather than creating a new instance.

Once selected init or clone, we get to the revision control template currently we have a choice of one : "git_ssh" - but hopefully more in future "git_https" and "batch_script" option are ones we would like to add - but for now select "git_ssh".

 

We now select a directory where we want to create the PolicyEnvRoot - We want the parent directory, for example the selection below is for "C:\work" (normally you would only have one PolicyEnvRoot directory, the extra ones you see here are the result from me running these tests) 

 

We then enter the name of the repository/path subdirectory name eg : 

 

We are then prompted for a Admin password for this PolicyEnvRoot - this password needed to create a new env, and is also used to store a copy of the xpsexport passwords for each environment so they can be recovered latter if needed. 

 

Finally we enter the path to the Git repository: 

 

And then we have the confirmation screen : 

 

And if we continue, we get the final screen confirming what directory is created and committed to the repository: 

 

After creation the current "Policy Env Root Settings"  can be viewed from "Options/Policy Root Env Settings" menu item: 

You can also verify the Admin password and reset it. 

 

(Note; the astute observers amongst you will have noticed some variation of the repository names and directories - yours should all be the same, these screenshots are an accumulation from a few sessions I had)

 

7.2  Create DEV Policy Store environment

 

The next step is to create a directory to store store our DEV policy store.  This is also done via a menu item off SMPolicyReader.  

 

Select the Options/New Env menu item.    

 

You are prompted for the PolicyEnvRoot Admin password, which you need to enter correctly before it will let you add a new environment entry.  

Then a short env code "DEV" is what we have entered here.  

You are then prompted for the password that will be used with XPSExport for this environment (each environment can have a different xpsexport password)..

Keeping an eye on the cmd line window for Git add commit and push commands, and then finally conformation the DEV environment is created and committed to revision control.

 

After creation of the DEV environment the setting can be viewed via menu item "Options/Policy Env Settings" : 

You can also check the XPSExport password, or recover it using the PolicyEnvRoot Admin password, or set it to a new value.

 

 

7.3  PolicyRootEnv and DEV env Directory Structure

At the end of that process we will have create an PolicyEnvRoot/ssoenv_DEV directory :

With some subdirectories roughly as seen here : 

 

The settings for the current PolicyEnvRoot and which environment are stored in the userSettings file (this is where the batch program will get its settings)  :

(There is a menu item on SMPolicyReader Options/Edit User Options that will open the file) 

 

 

8. Test run of ssobackup on SSO policy server.

 

The environment is now setup, so now we want to do a test run of the ssobackup process.  

 

From a DOS cmd window we cd to the ssobackup directory and run the program for the first time.

cd \apps\SMPolicyReader-4_0-ALPHA-426_bin\ssobackup

runBackup.bat

 

The ssobackup program runs XPSExport, manipulates and splits the export file and then commits the data to Git.  It writes the output to timestamped log file (it keeps 10 of them).  

 

The ssobackup_stdout_<timestamp>.log file can be checked to ensure the export and commit ran without problems.

 

I've tidied up most loose ends for adding files to the Git repository, but you can check the status of the created files via : 
    git status -s 

will give a good idea of any additional files that need to be committed : 

Also you can run other git commands, such as "git add" "git rm" etc,  to tidy up any extra files or directories you or the ssobackup program has added : "git log policystore.xml" show show the main revision history.

Once an environment is setup and running the files under revision control are usually fairly stable, and should not need many manual changes.  

 

9. Setup ssobackup program to run on policy server.

Now that our runBackup.bat is working the next step it to automate it, so that it runs every hour.   

 

9.1 On Windows 

On Windows we do that by adding a scheduled job.   From the "Start" menu search for the "Tesk Scheduler" program and start that. 

Here are the screenshots of the scheduled task that I have created : 

 

The overview: 

 

 

We set it up to run daily, and to repeat every 1 hours. 

 

The Action is to run the runBackup.bat script and we specify the startup directory as the same directory that the .bat file is in (in our case \apps\SMPolicyReader-4_0-ALPHA-426_bin\ssobackup) 

 

 

Once setup you can check the main "Task Scheduler" screen to see if the program has run, and also check the ssobackup_stdout_<timestamp>.log log files to verify that it is running correctly. 

 

 

9.2 On Linux 

On Linux you would setup the runBackup.sh to be run regularly via cron - with somethig like : 

 

You may need to tinker a bit so that it runs in the same directory (added a cd in the ssobackup directory for example).

 

 

10. View policy change history on dev policy server. 

 

The usage of the SMPolicyReader viewing history I will cover in another post.  

 

But for a quick sanity check, we can run the SMPolicyReader either select:

the menu : Env/"Open Policy Version History" 

right click on a Application and select "Show Version History"

 

You can then select two versions and hit compare. 

Hopefully you now see something similar to the above - if not then contact us via the page here give some details of the error you see, and we will see what we can do. 

 

11. Conclusion 

 

And that is it - we now have the DEV policy store being stored under revision control. 

 

You can repeat the above for each different environment QA, PROD etc.  You will need to add the ssoenv_QA etc directories to the existing PolicyEnvRoot Git Repository rather than creating a new one each time.   

 

For the stand alone developer, the setup is similar, except they do not have a separate environment and do not need to run the batch job. 

 

 

Cheers - Mark

----
Mark O'Donohue
Snr Principal Support Engineer - Global Customer Success

Statistics
0 Favorited
5 Views
0 Files
0 Shares
0 Downloads

Tags and Keywords

Comments

Oct 13, 2017 03:20 AM

I just want to say that all the information you have given here is awesome...great and nice blog thanks for sharing. And i hope this will be useful for many people..

Jun 01, 2017 09:42 PM

Just a note if you have trouble with the initial connection, via init or clone, check back in the DOS/shell window for the actual git command.   In once case it was simple mistyping of the git repository name  git@gitserver:PolicyEnvRootKim.git where as it should have been git@gitserver:/opt/git/PolicyEnvRootKim.git.   You can reset it from the command line (it had setup the wrong git remote settings) - but it was easier to deleted the local directoryand ran it again.  

 

Cheers - Mark

May 25, 2017 09:14 PM

 

git Incompatibilities

 

I had an internal request for problem when an older version of git was being used.   git 1.7.1 does not have --textconv on git cat-file command.  

 

The SMPolicyReader executes a child cmd line shell script to run git and xpsexport commands, the DOS/UNIX commands are held in a properties file, providing a different properties file was part we would allow different revision control systems to work, including one where it just runs a .bat or .sh file for each step. 

 

To allow slight changes in the cmd that is run, I've made a change to the SPPolicyReader/ssobackup to allow the settings for individual commands to be overwritten locally from the default values.  That allows us to work through this incompatabiltiy for git versions on the cat-file command,  and should allow working around similar problems.

 

gitCommands.properties 

The git Commands I use are stored in internal properties file, I have attached the internal one I use to the email (gitCommands.properties)  

 

For the SMPolicyReader program, I also made a quick change,

With new version: 443    https://communities.ca.com/thread/100333222

So you will need to download this latter version, which allows overwriting the default commands.

 

gitCommands_Local.properties 

After loading the default gitCommands.properties, it also will now check for a file : ~/.SMPolicyReader/conf/gitCommands_Local.properties and any values it finds in there will overwrite the ones from the std default file.

 

For this specific case, the default GitCatFile command from : gitCommands.properties is:  

runcmd_GitCatFile=git cat-file --textconv $VersionHash:./$Filename

runcmd_GitCatFile_Success=

runcmd_GitCatFile_Fail=error

 

The local gitCommands_Local.properties can overwrite that eg : 

runcmd_GitCatFile=git cat-file -p $VersionHash:./$Filename

#runcmd_GitCatFile=git cat-file --textconv $VersionHash:./$Filename

runcmd_GitCatFile_Success=

runcmd_GitCatFile_Fail=error

Which is acceptable syntax for git 1.7.1.  

 

The gitCommands_Local.properties file needs to be placed in the following directory: 

Unix : ~/.SMProperties/conf/

 Windows:  C:\Users\<user>AppData\Roaming\CA\smpolicyreader\conf

 

 To check, on load there should be a message in cmd stdout if it finds the local properties file : 

 

 

That method may also be helpful to overcome other incompatibility issues.

 

 

gitCommands.properties 

Here are all the commands in the default gitCommands.properties file that can be overwritten by adding entriy in local gitCommands_Local.properties file. 

 

runcmd_XpsExport=XPSExport -xb raw-backup.xml -pass $XPSExportPassword -f
runcmd_XpsExport_Fail=FATAL
runcmd_XpsExport_LogFile=Log output: (.*)$

#----

runcmd_GitInit=git init
#runcmd_GitInit_Success=
runcmd_GitInit_Fail=^fatal: (.*)$


#----

runcmd_GitInitAdd=git add $Filename
#runcmd_GitInitAdd_Success=
runcmd_GitInitAdd_Fail=error
runcmd_GitInitAdd_Fail=^fatal: (.*)$

#----

runcmd_GitInitCommit=git commit -am "initial commit"
#runcmd_GitInitCommit_Success=
runcmd_GitInitCommit_Fail=error
runcmd_GitInitCommit_Fail_1=^fatal: (.*)$


#----


runcmd_GitRemoteAdd=git remote add origin $RepositoryPath
#runcmd_GitRemoteAdd_Success=
#runcmd_GitRemoteAdd_Fail=^fatal: (.*)$

# This result happened on redo of commands after an error - but probably should be able to cater for it.
#Command :git remote add origin git@gitserver:/opt/git/PolicyEnvRoot5.git
#fatal: remote origin already exists.


#----

runcmd_GitInitPush=git push -u origin master
#runcmd_GitInitPush_Success=
runcmd_GitInitPush_Fail=error
runcmd_GitInitPush_Fail=^fatal: (.*)$


#----

runcmd_GitClone=git clone $RepositoryPath $Filename
#runcmd_GitClone_Success=
runcmd_GitClone_Fail=^fatal: (.*)$

 

#----

runcmd_GitStatus=git status -s
#runcmd_GitStatus_Success=

runcmd_GitStatus_Fail= M (\\S+)
runcmd_GitStatus_Fail_1=\\?\\? (\\S+)
runcmd_GitStatus_Fail_2= D (\\S+)
runcmd_GitStatus_Fail_3= A (\\S+)

runcmd_GitStatus_Modified= M (\\S+)
runcmd_GitStatus_Unknown=\\?\\? (\\S+)
runcmd_GitStatus_Deleted= D (\\S+)
runcmd_GitStatus_Added= A (\\S+)

#----

runcmd_GitStatusFile=git status -s $Filename
#runcmd_GitStatusFile_Success=

runcmd_GitStatusFile_Fail= M (\\S+)
runcmd_GitStatusFile_Fail_1=\\?\\? (\\S+)
runcmd_GitStatusFile_Fail_2= D (\\S+)
runcmd_GitStatusFile_Fail_3= A (\\S+)

runcmd_GitStatusFile_Modified= M (\\S+)
runcmd_GitStatusFile_Unknown=\\?\\? (\\S+)
runcmd_GitStatusFile_Deleted= D (\\S+)
runcmd_GitStatusFile_Added= A (\\S+)

#----

runcmd_GitAddFile=git add $Filename
#runcmd_GitAddFile_Success=nothing to commit
runcmd_GitAddFile_Fail=fatal:

#----

runcmd_GitDeleteFile=git rm $Filename
#runcmd_GitDeleteFile_Success=
runcmd_GitDeleteFile_Fail=fatal:

#----

runcmd_GitRenameFile=git mv $FromFile $ToFile
#runcmd_GitRenameFile_Success=
runcmd_GitRenameFile_Fail=fatal:


#----

runcmd_GitCommit=git commit -am "Policy updated from bg monitor service"
#runcmd_GitCommit_Success=
runcmd_GitCommit_Fail=error
runcmd_GitCommit_Fail_1=fatal:

#----

runcmd_GitPush=git push origin master
#runcmd_GitPush_Success=
runcmd_GitPush_Fail=error
runcmd_GitPush_Fail=^fatal:
# Also if need to refresh - ! [rejected] master -> master (fetch first)
# And if need to git pull - ! [rejected] master -> master (non-fast-forward)

#----

# Returns hash of changes commited but not yet pushed to original (ie usually connection to server is down)
runcmd_GitPushCheck=git cherry
runcmd_GitPushCheck_Success=^+ (\\S+)
runcmd_GitPushCheck_Fail=^(w+)$
runcmd_GitPushCheck_Fail=^fatal:

#----

runcmd_GitFetch=git fetch
runcmd_GitFetch_Success=
runcmd_GitFetch_Fail=error

#----

runcmd_GitPull=git pull origin master
runcmd_GitPull_Success=
runcmd_GitPull_Fail=error

#----
runcmd_GitLog_SelectOptions=-$NumRevisions
runcmd_GitLog_SelectOptions=--since="$timestamp"

runcmd_GitLogFile=git log --shortstat --pretty= "%n----%nVersionHash: %h%nCommitter: %cN <%ce>%nCommitDate: %cd [%cr]%nSubject: %s" $SelectOptions -- $Filename
#runcmd_GitLogFile_Success=
runcmd_GitLogFile_Fail=error

#----

runcmd_GitCatFile=git cat-file --textconv $VersionHash:./$Filename
runcmd_GitCatFile_Success=
runcmd_GitCatFile_Fail=error

 

 

Cheers - Mark

----
Mark O'Donohue
Snr Principal Support Engineer - Global Customer Success

Related Entries and Links

No Related Resource entered.