Notepad++ and Regular Expression: Convert case in LDIF files

[Alan_Baugher]

Team,

 

Recently had a request from a site, to help manage a case challenge with data within a userstore.  The userstore was being populated by a HR solution, via the IM bulk loader client. 

 

Challenge:  During initial loading and ongoing loading, 10% of the uid: strings were in upper case, and about 100% the managerid: strings is upper case.

 

The CA Identity Management/Portal solution are case-insensitive.  The login uid case becomes a challenges when integrating with the CA Identity Governance solution.  

 

 

 

Proposal:

 

We broke this challenges into two (2) areas:   initial data clean-up; and adjusting the onboarding business logic.

 

This discussion is how to use the Regular Expression within Notepad++ to rapidly search/replace on case and replace those values for a userstore's LDIF export.

- Note:  If the userstore is on a UNIX/Linux servers, it is also possible to do this process with sed, awk, or perl.

 

 

Steps:

 

Step 1:   Identify the strings that need to be updated, e.g.  uid:  ABC,   managerid:  DEF

 

Step 2:   Use a design tool for Regular Expression.

- Several are available but a useful one for Notepad++ was  https://regex101.com/

 

Step 3:  Introduce some sample data and then use the designer tool, to build the correct search pattern and the replacement pattern.   These will be used later in Notepad++.

 

Example:

 

 

Where the search pattern is:  (\buid: \b)([A-Z]+)

And the replacement patter is: \1\L\2\E

 

Another example:

 

 

Where the search pattern is:  (\buid: \b)(.[A-Z].)

And the replacement patter is: \1\L\2\E

 

 

Step 4:   After you are satisfied, copy these search/replacement patterns to your Notepad++ GUI; and then submit to update the LDIF exported file.

 

 

or

 

 

 

Step 5:  Import the updated file into the Directory solution, e.g. if CA Directory use either dxloaddb  or dxmodify command(s).

 

 

 

 

Let me know if you found this useful.

 

 

Cheers,

 

A.

[Edward Royston]

Or, my favourite - use a Perl Pie...

perl -p -i -e 'tr/A-Z/a-z/' file
It does what you need in one command, is very fast, can be done from the command line and is applied directly on the file without having to generate a new file (as in it does it in place)...

[Alan_Baugher]

Hi Edward,

 

Thanks for the reply.   Perl/Awk/Sed are very powerful and useful for string manipulations, I would agree.

 

Two (2) additional restraints to consider:

 

1) For selectively replacement, such as for an LDIF file, the example would need to be adjusted, so that not all characters are impacted.   

We must leave the other attributes as-is, e.g. sn, givenName, etc.

 

2) For customers/sites that do not have access to Unix/Linux OS nor the ability to install Perl on their workstation/servers, we will need to identify other options.

 

 

 

NotePad++ was one option the customer/site had access to; and was able to leverage the non-install/invasive selection:

 

 

https://notepad-plus-plus.org/repository/7.x/7.5/npp.7.5.bin.x64.zip 

 

The below download packages, are able to be used on workstations/servers, that have "locked down" group domain policies to prevent installations.

 

 

Cheers,

 

A.

[Edward Royston]

Alan,

 

Yes, getting the zip version of Notepad with no install is a great option (as discussed elsewhere on the forums).

 

However, I just want to correct the Perl Pie line for your situation. The example above would obviously work in converting a file from upper to lower case.

 

What you would need to do is this:

 

perl -p -i -e ‘s/^uid: (.*)/uid: \L$1/g' <filename>

 

That should go through your file and every line that begins with "uid: " move the rest of the line from any case over to lower case. It will leave all of the rest of the contents in the file alone (including the sn/givenName/dn fields…

 

If you wanted to move it all to upper case then change the \L flag to \U (as you can pick up in the regex checker. Alternatively, I could grab your match from the regex checker. However, I would possibly put in a check for white space before the "uid: " just in case the file that you have has been adjusted.

 

I am sure that it could also be done in PowerShell as well in the case where you couldn't even get a canned Notepad++ onto a system – but will leave that as to an "exercise for the reader".

 

Hey, it's Friday… 

 

Ed

[Patrick Dussault]

Hi,

 

Just to add that emacs 64 bit has embedded commands to converts characters, and to handle regular expressions. That's very helpful. Emacs on windows hasn't dependencies to the OS. No installation of the product, just download it and run it. And it's very old and stable software. I used it to open 1G file and convert characters strings quite fast.

 

Best Regards,

Patrick

[Pasquale_Russo]

Usually I also use UltraEdit... faster than Notepad++, able to open larger files... but yes, thanks for this input!

cheers,

Pasquale