Symantec IGA

Expand all | Collapse all

Automate backup of DSAs (IMPD/IMCD)

  • 1.  Automate backup of DSAs (IMPD/IMCD)

    Posted 09-10-2013 11:17 AM
    Hello All,
    You may find this of use. I use this process, in conjunction, with my IME backup process, to ensure I have daily snapshots of my environment and userstores.
    IME Backup: 98226450


    How to automate your IMCD (IM corporate user store) and IMPD (IM provisioning user store) for daily backup and extract to LDIF files; and remove files older than 14 days.


     
    1) Automate the dump dxgrid-db command in one file; instead of via console or in four files; using the timing feature of dump command & flexibility of dxserver config files.
     
     
    IMPD has 4 DSA, within all four (4) DXI files on each IMPD server; add the following line at the bottom:
     
    hostname-impd-main.dxi
    hostname-impd-co.dxi
    hostname-impd-inc.dxi
    hostname-impd-notify.dxi
     
    # set common back time
    source "../settings/impd-backup.dxc";
     
    Create the file “../dxserver/config/settings/impd-backup.dxc”
     
    impd-backup.dxc
    # CA Directory DXserver/config/settings
    # Backup of IMPD via DXGRID OFFLINE BACK PROCESS
    # This will start the backup at 11:30pm EDT and run every 24 hours
    # Note: the start time is the offset in seconds from midnight of UTC
    # For 23:30 EDT when need the time in UTC =>  23:30 EDT + 5 hours =  0430 UTC
    # 5 hour difference between ET and UTC;  4.5 * 3600 = 16200
    dump dxgrid-db period 16200 86400;
    # To dump immediately, uncomment the below line; and issue a dxserver init all command: VERY Useful
    # dump dxgrid-db;
     
     
    This one file will allow for one location for changing backup times for offline backups and ensure that ALL IMPD DSA are backed up at the same time.
    As dump uses UTC time, this works on IMPD servers across TimeZones on WANs.
     
    If you wish to create an offline backup immediately, you can adjust the time or uncomment the “dump dxgrid-db;” and then perform a “dxserver init all”
    Don’t forget to reset back, if you do this; otherwise you will see an offline file recreated every time the dxserver is restarted.
    FYI – If you have a slow enough system, you can see the zdb files be first created as zd$ until the file is complete.  So if you see a zdb extension, then the file is done updating.
     

     
    2) After the offline ZDB files are created, it is useful to have daily LDIF extracts of the data (IMCD/IMPD) to validate data is in sync.
    a. Use Windows Scheduler with an ADS Domain or Local account. This account may have standard user access to AD or local access.

     
     
    @echo on
     
    :: *** Set the date string for file naming and archive ***
    for /f "tokens=1,2" %%u in ('date /t') do set d=%%v
    set t=%time:~0,2%%time:~3,2%%time:~6,2%%time:~9,2%
    if "%t:~0,1%"==" " set t=0%t:~1%
    set datestr=%d:~6,4%%d:~0,2%%d:~3,2%_%t%
    echo %datestr%
      
    :: View IMPD DSA names
    dxserver status
     
    :: Backup the three primary IMPD DSAs
    dxdumpdb -z -f %datestr%_%COMPUTERNAME%-impd-main.ldif  %COMPUTERNAME%-impd-main
    dxdumpdb -z -f %datestr%_%COMPUTERNAME%-impd-inc.ldif  %COMPUTERNAME%-impd-inc
    dxdumpdb -z -f %datestr%_%COMPUTERNAME%-impd-co.ldif  %COMPUTERNAME%-impd-co
     
    :: Export to LDIF the below notify DSA if needed for troubleshooting
    dxdumpdb -z -f %datestr%_%COMPUTERNAME%-impd-notify.ldif  %COMPUTERNAME%-impd-notify
     
    :: Export to LDIF the Corporate User Store DSA
    dxdumpdb -z -f %datestr%_%COMPUTERNAME%-corp.ldif  %COMPUTERNAME%-corp

    :: Remove files older than 14 days; to prevent drive from filling up with files.
    forfiles /s /m  *.ldif  /d -14  /c "cmd /c del @file"
     
    3) If the corporate user store is on CA Directory , do the same steps for the corporate user store (IMCD) dxi file and windows scheduler.
    4) If the SiteMinder Server is on CA Directory, ditto.



    Additional Backup Notes:

    How to backup your Active Directory user store information using MS tools CSVDE/LDIFDE

    This is useful track down when provisioning updates occur to Active Directory.
    The MS tools of CSVDE/LDIFDE can be executed from the IMPS (IM provisioning server)
    The user information pulled from Active Directory will be public non-privacy information, e.g. User DN, User's SamAccountID, User's MemberOf (AD Groups) but NOT Password

    5) To provide a sanity check before using the AD endpoint or AD Account Templates; it is useful to perform a daily extract of the user objects in AD.
    a. This is especially useful during production rollout when minor sync issues may occur; and allows us to be proactively aware of any “mass” changes occur.
    b. Use Windows Scheduler with an ADS Domain account. This account may have standard user access to AD.


    :: Use MS ADS Resource Kit tool, ldifde and csvde to export ADS data
    :: to LIDF and CSV formatted files

    :: Before running this command replace the two (2) variables below
    :: Replace Hostname of an Active Directory Domain Controller Hostname if %USERDOMAIN% does not resolve.
    :: Replace ADSDOMAIN with the correct base DN syntax, e.g. "DC=corp,DC=company,DC=com"
    :: This program may be execute as any Active Directory User to pull public AD data on TCP Port 389.

    ::set HOSTNAME=%USERDOMAIN%
    set HOSTNAME=AD_DC_hostname_here
    set ADSDOMAIN="dc=corp,dc=company,dc=com"
    ::set ADSDOMAIN=%USERDNSDOMAIN%

    :: SET DATESTAMP / TIMESTAMP
    :: *** Set the date string for file naming and archive ***
    for /f "tokens=1,2" %%u in ('date /t') do set d=%%v
    set t=%time:~0,2%%time:~3,2%%time:~6,2%%time:~9,2%
    if "%t:~0,1%"==" " set t=0%t:~1%
    set datetimestamp=%d:~6,4%%d:~0,2%%d:~3,2%_%t%
    @echo %datetimestamp%
    @echo.
    ::set hostname=%COMPUTERNAME%
    @echo %hostname%
    @echo.

    :: CSV Extract
    csvde -f %datetimestamp%_ADS_Export_Users.csv -s %HOSTNAME% -d %ADSDOMAIN% -p subtree -r "(&(objectCategory=person)(objectClass=User)(displayName=*))"
    @echo Full CSV Extract of AD Users Complete
    :: pause

    :: LDIF Extract
    ldifde -f %datetimestamp%_ADS_Export_Users.ldif -s %HOSTNAME% -d %ADSDOMAIN% -p subtree -r "(&(objectCategory=person)(objectClass=User)(displayName=*))"
    @echo Full LDIF Extract of AD Users Complete

    :: pause
    :: Remove files older than 14 days; to prevent drive from filling up with files.
    forfiles /s /m *.ldif /d -14 /c "cmd /c del @file"
    forfiles /s /m *.csv /d -14 /c "cmd /c del @file"


  • 2.  RE: Automate backup of DSAs (IMPD/IMCD)

    Posted 09-10-2013 01:56 PM
    Thanks for sharing this great information! :grin:

    baual01 wrote:

    Hello All,
    You may find this of use. I use this process, in conjunction, with my IME backup process, to ensure I have daily snapshots of my environment and userstores.
    IME Backup: 98226450


    How to automate your IMCD (IM corporate user store) and IMPD (IM provisioning user store) for daily backup and extract to LDIF files; and remove files older than 14 days.


     
    1) Automate the dump dxgrid-db command in one file; instead of via console or in four files; using the timing feature of dump command & flexibility of dxserver config files.
     
     
    IMPD has 4 DSA, within all four (4) DXI files on each IMPD server; add the following line at the bottom:
     
    hostname-impd-main.dxi
    hostname-impd-co.dxi
    hostname-impd-inc.dxi
    hostname-impd-notify.dxi
     
    # set common back time
    source "../settings/impd-backup.dxc";
     
    Create the file “../dxserver/config/settings/impd-backup.dxc”
     
    impd-backup.dxc
    # CA Directory DXserver/config/settings
    # Backup of IMPD via DXGRID OFFLINE BACK PROCESS
    # This will start the backup at 11:30pm EDT and run every 24 hours
    # Note: the start time is the offset in seconds from midnight of UTC
    # For 23:30 EDT when need the time in UTC =>  23:30 EDT + 5 hours =  0430 UTC
    # 5 hour difference between ET and UTC;  4.5 * 3600 = 16200
    dump dxgrid-db period 16200 86400;
    # To dump immediately, uncomment the below line; and issue a dxserver init all command: VERY Useful
    # dump dxgrid-db;
     
     
    This one file will allow for one location for changing backup times for offline backups and ensure that ALL IMPD DSA are backed up at the same time.
    As dump uses UTC time, this works on IMPD servers across TimeZones on WANs.
     
    If you wish to create an offline backup immediately, you can adjust the time or uncomment the “dump dxgrid-db;” and then perform a “dxserver init all”
    Don’t forget to reset back, if you do this; otherwise you will see an offline file recreated every time the dxserver is restarted.
    FYI – If you have a slow enough system, you can see the zdb files be first created as zd$ until the file is complete.  So if you see a zdb extension, then the file is done updating.
     

     
    2) After the offline ZDB files are created, it is useful to have daily LDIF extracts of the data (IMCD/IMPD) to validate data is in sync.
    a. Use Windows Scheduler with an ADS Domain or Local account. This account may have standard user access to AD or local access.

     
     
    @echo on
     
    :: *** Set the date string for file naming and archive ***
    for /f "tokens=1,2" %%u in ('date /t') do set d=%%v
    set t=%time:~0,2%%time:~3,2%%time:~6,2%%time:~9,2%
    if "%t:~0,1%"==" " set t=0%t:~1%
    set datestr=%d:~6,4%%d:~0,2%%d:~3,2%_%t%
    echo %datestr%
      
    :: View IMPD DSA names
    dxserver status
     
    :: Backup the three primary IMPD DSAs
    dxdumpdb -z -f %datestr%_%COMPUTERNAME%-impd-main.ldif  %COMPUTERNAME%-impd-main
    dxdumpdb -z -f %datestr%_%COMPUTERNAME%-impd-inc.ldif  %COMPUTERNAME%-impd-inc
    dxdumpdb -z -f %datestr%_%COMPUTERNAME%-impd-co.ldif  %COMPUTERNAME%-impd-co
     
    :: Export to LDIF the below notify DSA if needed for troubleshooting
    dxdumpdb -z -f %datestr%_%COMPUTERNAME%-impd-notify.ldif  %COMPUTERNAME%-impd-notify
     
    :: Export to LDIF the Corporate User Store DSA
    dxdumpdb -z -f %datestr%_%COMPUTERNAME%-corp.ldif  %COMPUTERNAME%-corp

    :: Remove files older than 14 days; to prevent drive from filling up with files.
    forfiles /s /m  *.ldif  /d -14  /c "cmd /c del @file"
     
    3) If the corporate user store is on CA Directory , do the same steps for the corporate user store (IMCD) dxi file and windows scheduler.
    4) If the SiteMinder Server is on CA Directory, ditto.



    Additional Backup Notes:

    How to backup your Active Directory user store information using MS tools CSVDE/LDIFDE

    This is useful track down when provisioning updates occur to Active Directory.
    The MS tools of CSVDE/LDIFDE can be executed from the IMPS (IM provisioning server)
    The user information pulled from Active Directory will be public non-privacy information, e.g. User DN, User's SamAccountID, User's MemberOf (AD Groups) but NOT Password

    5) To provide a sanity check before using the AD endpoint or AD Account Templates; it is useful to perform a daily extract of the user objects in AD.
    a. This is especially useful during production rollout when minor sync issues may occur; and allows us to be proactively aware of any “mass” changes occur.
    b. Use Windows Scheduler with an ADS Domain account. This account may have standard user access to AD.


    :: Use MS ADS Resource Kit tool, ldifde and csvde to export ADS data
    :: to LIDF and CSV formatted files

    :: Before running this command replace the two (2) variables below
    :: Replace Hostname of an Active Directory Domain Controller Hostname if %USERDOMAIN% does not resolve.
    :: Replace ADSDOMAIN with the correct base DN syntax, e.g. "DC=corp,DC=company,DC=com"
    :: This program may be execute as any Active Directory User to pull public AD data on TCP Port 389.

    ::set HOSTNAME=%USERDOMAIN%
    set HOSTNAME=AD_DC_hostname_here
    set ADSDOMAIN="dc=corp,dc=company,dc=com"
    ::set ADSDOMAIN=%USERDNSDOMAIN%

    :: SET DATESTAMP / TIMESTAMP
    :: *** Set the date string for file naming and archive ***
    for /f "tokens=1,2" %%u in ('date /t') do set d=%%v
    set t=%time:~0,2%%time:~3,2%%time:~6,2%%time:~9,2%
    if "%t:~0,1%"==" " set t=0%t:~1%
    set datetimestamp=%d:~6,4%%d:~0,2%%d:~3,2%_%t%
    @echo %datetimestamp%
    @echo.
    ::set hostname=%COMPUTERNAME%
    @echo %hostname%
    @echo.

    :: CSV Extract
    csvde -f %datetimestamp%_ADS_Export_Users.csv -s %HOSTNAME% -d %ADSDOMAIN% -p subtree -r "(&(objectCategory=person)(objectClass=User)(displayName=*))"
    @echo Full CSV Extract of AD Users Complete
    :: pause

    :: LDIF Extract
    ldifde -f %datetimestamp%_ADS_Export_Users.ldif -s %HOSTNAME% -d %ADSDOMAIN% -p subtree -r "(&(objectCategory=person)(objectClass=User)(displayName=*))"
    @echo Full LDIF Extract of AD Users Complete

    :: pause
    :: Remove files older than 14 days; to prevent drive from filling up with files.
    forfiles /s /m *.ldif /d -14 /c "cmd /c del @file"
    forfiles /s /m *.csv /d -14 /c "cmd /c del @file"