UCYBCRYP.EXE

[Michael A. Lowry]

I recently developed a shell script to generate Automation Engine INI files automatically, based on contextual information like server name and staging environment. This script allows us to guarantee that the only differences in the INI files between our various Automation Engine servers are differences that need to be there. This in turn helps us maintain the consistency of our DEV, TEST, and PROD servers.

The one part of this process that I have not figured out how to automate is encryption of the DB passwords.

We store all passwords in a central password vault. Authenticated and authorized users (including technical accounts) are permitted to retrieve passwords from the vault. When building an AE INI file, we could simply extract the password and insert it into the ODBC connection string in plaintext form. However this is risky, and we prefer to encrypt the password first. Right now, this is a manual step. We must install UCYBCRYP.EXE on a Windows PC, run it manually, and then copy the encrypted password to the INI file.

We would like to find a way to automate the process of encrypting Automation Engine DB passwords so that we can fully automate the process of generating the INI files. Is there a Java or REST API we can use to encrypt the password? Or would someone be willing to describe the algorithm?

[Nicole Boeger]

Dear Michael,

as far as I know, there is only this Windows-Exe-Tool to encrypt the password.

I have another idea: why don't you store the encrypted password in the password vault? That way you still need to execute the exe by hand and store it in the password vault, but you don't have a manual step when running the automated process.


------------------------------
Regards, Nicole
------------------------------

[Michael A. Lowry]

@Nicole Boeger wrote:

[W]hy don't you store the encrypted password in the password vault?

We cannot set the password. It is generated by the vault.

[Nicole Boeger]

Well, is that password vault changing the password regularly?

You could automatically run the ucybcryp.exe by an windows agent and store the encrypted password in a safe place?

Also: instead of coding a shell script, I would recommend to use a configuration tool like puppet, if you have the chance to. That's what I do. Once there is a puppet server in place, it is not much work to configure the ini files for the agents for example.

------------------------------
Regards, Nicole
------------------------------

[Michael A. Lowry]

We won't need to generate these INI files very often, but I want to be able to fully automate the process anyway. The script is part of a larger process that installs an entire AE server from scratch. The password vault is the safe place where passwords are stored. We do not want to store unencrypted passwords anywhere else. Yes, we could run a job on a Windows agent to run UCYBCRYP.EXE; this seems so kludgy though. If we had an API for this, or if we knew the algorithm, we could avoid relying on Windows.

We use Puppet for other things but it seems like overkill for this use case.  Moreover, presentations given by Broadcom suggest that the next version of Automic Automation will enable other industry-standard ways of automating installation & configuration of the application. So I do not want to invest too much time in this. (I know I whatever I work I do now will have to be redone in a year or so.)

But just out of curiosity, do you use Puppet today to configure your AE server INI files?

[Carsten Schmitz]

FWIW:

I have asked Automic/CA about their password algorithm previously. They very likely won't disclose it, it's security by obscurity. I agree there should be a text-based API though. Probably an "idea".

We use Puppet here, but not for Automic. Puppet's nature of overwriting things rather rigorously is annoying (though not sure if that is the only way to use it, or how people use it here). We used to build package manager packages for agents, but we abandoned that. Overkill as well, and no support from Automic. I now have a shell script that installs and configures smgr and agents on Linux, but the config side mostly consists of copying from a set of pre-prepared files into place.

If I were to use a tool for configuring stuff automatically, I'd look into Ansible, not Puppet. I believe Ansible is stronger with flexibly inserting config strings into files than Puppet is.

Hth,
​

[Nicole Boeger]

@Carsten Schmitz:
I guess, it depends on how you use puppet. I configured puppet only for initial values. For example, if there is no ini​​ file, it would be created by puppet with UNIX.<hostname> as a configured value for the agent name. Afterwards, when the agent starts, the agent itself changes the ini file by adding the ports of all CPs, so I rather not let both systems switch back and forth.
You also can use puppet to enforce values of single values within the ini file, like the hostname. If you want to enforce, that the agent name should never be changed, you can configure puppet to check regularly and change the value if it is not as expected. But you need to be aware of what effects that has.

------------------------------
Regards, Nicole
------------------------------

[Carsten Schmitz]

Thanks, good to hear Puppet has that capability also, like Ansible Playbooks.

I am probably just predisposed by the way our folks use puppet, which seemingly is to identify any changes I make to any UNIX system, then undo them :D

[Michael A. Lowry]

@Carsten Schmitz wrote:

I have asked Automic/CA about their password algorithm previously. They very likely won't disclose it, it's security by obscurity. I agree there should be a text-based API though. Probably an "idea".

I've noticed that if one runs UCYBCRYP.EXE multiple times with the same input text, the output text differs between runs. The beginning is the same, but the end differs. It does not seem to be a particularly complicated algorithm. It might not be difficult to reverse engineer.

[Carsten Schmitz]

It does not seem to be a particularly complicated algorithm. It might not be difficult to reverse engineer.

I had the same thought in 2016/2017, but I tried and couldn't figure it out. Which possibly makes the following sound arrogant, but I'll say it anyway: it also didn't exhibit the hallmarks of a very secure algorithm to me and reaked of "roll your own crypto".

Haven't looked into it since. Also, I thought they wanted to move to more modern and open crypto, didn't they announce that at some point? But hearing that ucybcryp is still in use, that probably didn't apply to the db password?

[Michael A. Lowry]

Based on a couple dozen passwords, here is a rudimentary analysis of the algorithm used by UCYBCRYP.EXE .

Basic observations

• The encrypted password always begins with the header ‘--10’: two soft hyphens, followed by '10'. This header is not part of the ciphertext.
• The remainder of the encrypted password is the ciphertext.
• The ciphertext is comprised of between one and five 16-character blocks of hexadecimal values. (Each block is exactly 256 bits.)
• The ciphertext is divided into two parts I will label part 1 and part 2.
• Part 1 is always included. Part 2 is included only for longer passwords.

Plaintext length	Part 1 length	Part 2 length
1–7	16	0
8–15	16	16
16-23	32	16
24–31	48	16
32	64	16

Part 1

• Probably DES in ECB mode.
• Deterministic: for a given plaintext string, part 1 will always be same regardless of how many times the program is run.
• The length of depends on the length of the plaintext. The program accepts plaintext of up to 32 characters in length.
• Subdivided into 1-4 blocks 16 hex digits (256-bits) in length.
• The blocks of correspond directly to chucks of the original plaintext.

Part 2

• A more modern algorithm, perhaps AES.
• Non-deterministic: the ciphertext differs for each time the program is run with the same input text.
• Always exactly one block (16 characters) in length.

General inferences

It's possible to guess quite a bit about the algorithm based just on these observations.

• There are actually two different algorithms in use. Algorithm 1 generates part 1, and algorithm 2 generates part 2.
• For short passwords (up to 7 characters), only algorithm 1 is used.
• For longer passwords (8 to 32 characters), both algorithm 1 and algorithm 2 are used and the outputs are concatenated.

Conjecture

Why are there two algorithms, and why does Algorithm 2 come into play only for longer passwords? One possible reason is backward compatibility. If early versions of the program supported only short (<8 character) passwords, and support for longer passwords was added only later, it’s possible that the two modes of operation were devised as a way to add support for longer passwords while still preserving the ability to use passwords encrypted with older versions of the program. Short passwords use the old way, and longer passwords combine the old way with a new way. Programs like ucybdbld that need to be able to decrypt both old-style and new-style passwords can distinguish the two styles based only on their length. This is just a guess though.

[Michael A. Lowry]

On a hunch, I just tried logging in to an AE system using a password encrypted using UCYBCRYP.EXE. It worked both via the AWI and the Java Application Interface.

This strongly suggests one of these possibilities:

• the AE server is able to decrypt passwords; or
• both the AWI and uc4.jar have this capability.

The AE DB Utility programs such as ucybdbld communicate directly with the AE database, so at least these programs do not rely on the AE server processes. The shared library libzugss.so used by these programs contains many mentions of AES, so perhaps this algorithm is used (considering other adjacent text, likely in CBC mode).

[Michael A. Lowry]

The whole point of encrypting passwords before storing them in INI files is to make these passwords unusable (or at least more difficult to use) for any unauthorized persons who manage to get hold of them.

However, if an encrypted password can be used to log in via the AWI or Java Application Interface, then the protection ostensibly provided by the encryption is completely nullified.

Am I overlooking something?

[Pete Wirfs]

"However, if an encrypted password can be used to log in via the AWI or Java Application Interface, then the protection ostensibly provided by the encryption is completely nullified."

True.  But in practice, when do you store a plain-text encrypted password for a userid that one could use to log in to UC4?  Off the top of my head, I can't think of any instances here.

------------------------------
Pete Wirfs
SAIF Corporation
Salem Oregon USA
------------------------------

[Nicole Boeger]

In my understanding, this encryption algorithm is used also in the login objects. The encrypted passwords are stored in the automic database and need to be decrypted by the engine when logging on as a user on the target OS or database. I think, the decryption need to be done by the engine processes.

When using &UC_JOBMD in a script, the passwords look very similar to the output of UCYBCRYP.EXE.

 

I bet, this encryption tool is a fast made shot and just intended to be used for the database login of the engine to avoid a clear text password in the ini file.

 

Regards,

Nicole

 

[Pete Wirfs]

When I run this script on a windows agent;

:print "uc_jobmd=&UC_JOBMD "​

​It returns;

uc_jobmd=C:\Automic\Agents\Windows\bin\UCXJWX6M.EXE​

------------------------------
Pete Wirfs
SAIF Corporation
Salem Oregon USA
------------------------------

[Michael A. Lowry]

@Pete Wirfs: I believe @Nicole Boeger means running the Agent Job Messenger in an OS job to decrypt a password retrieved from a LOGIN object using GET_LOGIN. If the topic seems familiar, you may recall that we discussed this 5 years ago. :)

The algorithm used to encrypt the passwords returned by GET_LOGIN seems to be DES, but the key is different. It's surely somewhere in the ucxjlx6m binary.

[Michael A. Lowry]

@Michael A. Lowry wrote:
This strongly suggests one of these possibilities:

• the AE server is able to decrypt passwords; or
• both the AWI and uc4.jar have this capability.

I confirmed that the client is doing the decryption.

In an experimental AE system, I created a test user with the password '--10abcdef'. I then tried to log in to the AWI using this user. The following message appeared immediately:

java.lang.RuntimeException: javax.crypto.IllegalBlockSizeException: Input length not multiple of 8 bytes

I tried a few more passwords beginning with variations on '--10', but only passwords beginning with this exact string triggered the exception. The AWI assumes any password beginning with '--10' is an encrypted password.

Behind the scenes, the AWI uses the Java Application Interface (uc4.jar) for most interaction with the AE server. With a bit of digging, starting with the Conection.login method, I quickly identified the class that performs the relevant operations.

🇭🇺🦒

[Nicole Boeger]

@Michael A. Lowry:
No, I configured just the unix agent ini files in puppet. Even the windows ini files I have not configured with puppet yet, because puppet is not running on our windows servers yet (it's hard to get the unix and windows team cooperate in this area).

We are still a pretty small shop (I am the only one administering and coding Automic), so we don't install new engines very often.

------------------------------
Regards, Nicole
------------------------------

[Michael A. Lowry]

I would now like to make a few more observations based on my investigations into this topic:

1. The potential benefits of encrypting passwords are not realized if encrypted passwords can be used in places like the AWI and Java Application Interface. For this reason, DB users should be dedicated to that role, and should not have other access rights. (Thanks, @Pete Wirfs.)
2. The added security of a more modern encryption algorithm is undermined if ciphertext generated by an old & insecure algorithm is still included, and if the ciphertext generated by the more modern algorithm can simply be omitted and the password will still work.
3. Security through obscurity is not real security.

[MATTHEW Selke]

This might be of interest.   I used ucybcryp.exe to create encrypted password for my AE DB and it worked fine in ucsrv.ini to connect to DB and start the AE's. 
However when I put the same encypted password in a variable to be used by &UC_JOBMD in a script I got error.  (in both V12.2.5 and V12.2.6) 

sh: -c: line 0: unexpected EOF while looking for matching `''
sh: -c: line 1: syntax error: unexpected end of file

I opened support case and after 6 weeks finally received the resolution.   Create a promptset add it to a script and execute.  Enter the password and output in report is the encrypted password for use with &UC_JOBMD.  

My observations are
1) running UCYBCRYP.EXE with the same password yeilds different result --10***  every time. 
2) Running promptset with that same password gives a different result to UCYBCRYP.EXE
3) Running promptset multiple times gives same result each time.

In my opinion it would be better to have a global encryption that does not rely on Windows.  Would have saved me at least 8 weeks of uncertainty. 

Regards,
Matt 

[Michael A. Lowry]

As I mentioned above, although the algorithm is similar and the output looks similar, the key used by the job messenger to decrypt passwords is different than the one used by UCYBCRYP.EXE to encrypt them.

[Michael A. Lowry]

A few days ago while looking into the HashiCorp action pack and its prerequisite the ITPA shared action pack, I identified another key used by the Automation Engine. This key is used to encrypt passwords in at least a couple of contexts:

• passwords from LOGIN objects in transit between the Automation Engine and the agent.
• values submitted for PRPT variables that have the Show as password check box enabled.

These passwords can also be decrypted in a Windows or Linux job using the Agent Job Messenger.

I also confirmed that the algorithm is the same one as used by UCYBCRYP.EXE: DES in ECB mode.

I identified a third key as well, but I have not yet confirmed in what context(s) it is used. (Maybe passwords from connection objects?)

It's no wonder that Broadcom has taken to calling this merely password obfuscation; the algorithm is ancient and the keys are trivial to find. New in the v21 documentation is this disclaimer:

Obfuscating passwords hinders easy disclosure of the values but does not guarantee confidentiality.

[Michael A. Lowry]

One additional data point:

It does not matter whether the dash characters are 0x2D or 0xAD. The programs that read encrypted passwords appear to treats both the same.

[Joachim_Zientek_934]

To bring the post up again :-)

Why do you not use WINE?
With that I was able to run Windows EXE on Linux or Mac and use UCYBCRYP.EXE also in non Windows environments.

The effort for reengineering is to high and error prone. Would give hime a try.

[Michael A. Lowry]

Emulation might have been an interesting work-around, but we found a more straightforward approach.

Having identified both the algorithm and the key, we wrote our own program to perform the necessary operations.