Timestamp password for Service Manager

[Michael A. Lowry]

Knowledge base article KB224476 documents a bug (AE-26648) fixed in 12.3.6 HF2. The problem is that the Automation Engine uses an undocumented password based on the UTC timestamp to control its own processes via the Service Manager. Someone with knowledge of the format of this password could also control the AE Server processes.

The format of the the timestamp-based password is described with a Windows PowerShell example:

$((Get-Date).ToUniversalTime().ToString("yyyMMdd$([char]0x01)HHmmss"))

I was able to reproduce the described behavior. Below is a summary of my tests. All tests were conducted using UCYBSMCL.EXE on a Windows VM.

Service Manager version	TLS 1.2 secured	Hosted service & version	Timestamp-based password accepted
12.3.6	✘	Automation Engine 12.3.6	✔
12.3.6	✔	Automation Engine 12.3.6	✔
12.3.6	✘	Automation Engine 12.3.6 HF3	✔
12.3.6	✔	Automation Engine 12.3.6 HF3	✔
12.3.6 HF2	✘	Automation Engine 12.3.6	✘
12.3.6 HF2	✔	Automation Engine 12.3.6	✘
12.3.6 HF2	✘	Automation Engine 12.3.6 HF3	✘
12.3.6 HF2	✔	Automation Engine 12.3.6 HF3	✘
12.3.6	✘	UNIX Agent 12.3.6	✔
12.3.6	✔	UNIX Agent 12.3.6	✔
12.3.6	✘	UNIX Agent 12.3.6 HF3	✔
12.3.6	✔	UNIX Agent 12.3.6 HF3	✔
12.3.6 HF2	✘	UNIX Agent 12.3.6	✘
12.3.6 HF2	✔	UNIX Agent 12.3.6	✘
12.3.6 HF2	✘	UNIX Agent 12.3.6 HF3	✘
12.3.6 HF2	✔	UNIX Agent 12.3.6 HF3	✘

The fix for AE-26648 in 12.3.6 HF2 definitely changes the behavior. Under the fix description, the only changed component listed is the Automation Engine. It seems obvious from the above results however that the only thing that matters is whether the Service Manager has been updated to 12.36 HF2.

Whether TLS is enabled, and whether the hosted service (AE or Agent) has also been updated, appear to be irrelevant to whether the timestamp-based password is accepted. (In an earlier revision to this post, I reported that in one scenario even the updated 12.3.6 HF2 Service Manager accepted the timestamp-based password. This was incorrect.)

It also seems clear that both the AE Server and the Service Manager would have to be updated to change the behavior without breaking things. I have not tested yet in what configurations it is possible to use the Automation Engine sheet of the Administration perspective in the Automic Web Interface to stop & start AE Server processes.

Perhaps the updated Automation Engine uses a different undocumented internal password that is harder to guess. Perhaps the Automation Engine uses its back-channel to the Service Manager (specified via the -svc command line option for the CP & WP). In any event, I get the impression that 12.3.6 HF2 represents only the first part of what will eventually be a more thorough fix.

One can also reproduce the behavior using the ucybsmcl binary on Linux:

"$(date -u +%Y%m%d)$(printf '\x01')$(date -u +%H%M%S)"

In ASCII, x01 is the SOH (Start of Header) character. Thanks to @Ivan Barumov, who discovered how to get it working on Linux.

Any additional information would be appreciated.​​

[Michael A. Lowry]

With additional testing, I confirmed my suspicion.

Once you have updated the Service Manager that hosts the Automation Engine to 12.3.6 HF2, you will no longer be able to start these server processes using the Automation Engine sheet of the Administration perspective in the Automic Web Interface. Attempts to start AE server processes via the updated Service Manager 12.3.6 HF2 will result in the error message:

Service Manager reports unknown error, error code '112'.

This is essentially the same error that appears when trying to start the process using UCYBSMCL.EXE and specifying the timestamp-based password.

*ERROR 3* Protocol violation (112)

ERROR 3 is what one gets if one specifies an invalid password for an operation that requires a password.

I imagine that the capability to start processes will be restored in a future update to the SMgr/AE/AWI.

[Keld Mollnitz]

Hi @Michael A. Lowry,

It's always interesting to follow your posts in the communities. Keep going !​

/Keld.

[Michael A. Lowry]

I received an update from Broadcom on this question. The support specialist helpfully pointed me to a part of the documentation I had not seen before. At the end of the page describing the Service Manager - Dialog Program, there is a section describing how to tell the Automation Engine & Automic Web Interface what passwords to use when contacting Service Managers.

The documentation is characteristically unclear, so I did some tests to see how it works. Below are my instructions.

---

UC_SMGR_LOGINS

Service Manager passwords can be stored a special LOGIN object called UC_SMGR_LOGINS in client 0.

To store the password to use when communicating with an Agent Service Manager, select the name of the agent in the Agent / Name field.
To store the password to use when communicating with the Automation Engine Service Manager, insert the placeholder * into the Agent / Name field and the Type field.

In both cases, the Username / ID is irrelevant. You must put a value in this field, but the value does not matter.
The password should be the password (L2 or L3) of the corresponding Service Manager.

Here's an example.

In this example, two Service Manager passwords are stored:

• The password for controlling AGENT123 via its Service Manager;
• The password for controlling the Automation Engine server processes via the AE Service Manager.

There is no way to specify a wildcard to indicate a group of agents. If you have 1000 agents and they all use the same Service Manager password, you must unfortunately add 1000 agent entries to to login object.

---

Additional details

The UC_SMGR_LOGINS object (ID 50048) is included in the AE 12.3.6 HF2 initial data. It appears in client 0 under the folder DIV_VARIABLES (folder ID 822).

F001+00000
F002CLOGIN
F003CUC_SMGR_LOGINS
F004+0000050048
F005CStores the SMGR login
F006+0000000005
F00720200726092618
F00920210914102740
F010+0000000001
F063C00000000000000000000000000000000
F072C11.2
R

This object is not present in the AE 12.3.6 initial data. I gather the documentation was updated for 12.3.6 HF2 as well.

We can therefore conclude that the complete fix for AE-26648 in 12.3.6 HF2 encompasses at least these four changes:

• Service Manager: close timestamp password backdoor;
• Automation Engine: add ability to use Service Manager passwords stored in UC_SMGR_LOGINS;
• Initial Data: add UC_SMGR_LOGINS object;
• Documentation: Add description of UC_SMGR_LOGINS object and how to use it.

I hope this helps clarify the matter.

I trust I am not the only one who believes it would be preferable if Broadcom made a effort to proactively publish complete and timely descriptions of vulnerabilities and fixes.

[Carlos Cardano]

Hello Michael,

Thanks for posting this topic as we are also in the middle of this topic.

Just want to ask you if or how you are managing the password for each service manager since changing password via command line is not yet available and we are talking 15000+ Agents.

Scenario is you have a third party application that monitors and starts your Windows Agent when tagged as "Automatic Start" and for this, you need password.

Now, we want to provide only the password_l2 to the third party and the password_l3 for us admin.

But you can only change password via Service Manager Dialog.

Have you also seen this scenario and best practice on handling this password?

Thanks for the reading this comment.





------------------------------
Regards,
Carlos
------------------------------

[Marcin Uracz]

Hi Carlos,
I will chip in. 

The passwords in the ini file are, based on my research/tests, SHA-512 hashes with no salt. So you can generate one with simple linux command:

echo -en test| shasum -a 512​

And then put it in the right place using sed for example. No restart of the SMGR is necessary.

------------------------------
Cheers,
Marcin
------------------------------

[Carlos Cardano]

Hi Marcin,
Thanks for your input.
Can you share also how it is done via Windows command prompt?
​

------------------------------
Regards,
Carlos
------------------------------

[Ivan Barumov]

I would recommend a docu object  to store the ini file/s  and run this true the system via simple job on OS level.
Whenever password change is needed create new one and replace the hashed string in the docu object and run the job again
Loop true the Docu with PREP_PROCESS_DOCU and write it to file on the agent WRITE_PROCESS
This runs no matter of the OS or version of it. Runs like a charm and it can be customized and automated as much as anyone likes

[Christoph Rekers]

Hi @Carlos Cardano,

I just stumbled over this thread​... This works for me to hash the SMGR Password with Powershell:

$mystring = "my level-2 password"
$mystream = [IO.MemoryStream]::new([byte[]][char[]]$mystring)
Get-FileHash -InputStream $mystream -Algorithm SHA512 | Format-Table -AutoSize


Cheers
Christoph

------------------------------
----------------------------------------------------------------
Automic AE Consultant and Trainer since 2000
----------------------------------------------------------------
------------------------------

[Michael A. Lowry]

@Marcin Uracz:  Good find. I knew it was different from the algorithm used by UCYBCRP.EXE but I didn't realize it was just a hash.

This means it would be straightforward to automatically generate ucybsmgr.ini files as well as insert the corresponding passwords into the UC_SMGR_LOGINS object, e.g., with the LoginDefinition Java API class.

[Ivan Barumov]

Thank you Michael,

I can confirm that this works the same from unix ucybsmcl

Changing the '\x0001' to '\x01'  did the trick not sure why as the printed in file characters are the same SOH but it does not matter in this case.

So in theory to get this particular vulnerability fixed for the moment only the service manager can be updated as in any case braking the functionality  from the AWI.

 

[Michael A. Lowry]

I just discovered a quick way to test Service Manager passwords that is non-destructive. I found that ucybsmgr checks any provided password even if the requested operation does not require a password. This means you can test a password by submitting a command like GET_PROCESS_LIST that normally does not require a password.

Details
In ucsybsmgr.ini, there are three different passwords:

• password_l1: allows read-only operations like GET_PROCESS_LIST
• password_l2: allows L1 operations plus change operations like START_PROCESS and STOP_PROCESS
• password_l3: allows L1 & L2 operations plus admin operations like SET_DATA

Typically, no L1 password will be set, which means that you do not need to provide a password just to run the GET_PROCESS_LIST command. I found however, that in this case if you use ucybsmcl to send the GET_PROCESS_LIST command, but specify a password anyway, the Service Manager will still check the password even though a password is not required.

Passwords that are accepted:

• The L2 or L3 password.
• The timestamp-based password (prior to SMgr 12.3.6 HF2)