Clarity Service Management

SpelMaster Event

By Giedrius Bekintis posted 06-08-2016 05:38 PM

  

Hello Community,

Recently I have discovered how to overcome one of the most frustrating limitations of CA SDM Spel Code. The “limitation” in mind was very annoying and interfered with some of my ideas on how to improve CA SDM. It took me about 10 years and countless hours to finally overcome it

Of course I would like to share this with You guys. But … this isn’t something that I would like to give up so easily . I had a chat with Chris and we agreed to organize a “friendly competition” in which you will have a chance to find by Yourself solution for this “limitation”. The first person that will get it working will receive special SpelMaster badge. If nobody will manage, this secret will remain unrevealed.

Rules for our “friendly competition”: * Please review a short video demo provided bellow and identify what Spel Code limitation had to be solved in order to develop such functionality. Your valid guesses can be posted as comments. The correct guess will be a trigger for the next stage of competition.

This first step is intended to see if CA Service management community is interested in my findings, so if no correct answer is given within 10 days, we’ll end this all together.

* After we get correct identification of the problem to solve, I will provide an assignment to solve and the first clue that will help You do that. If no solution is provided after 5 days, I will post another clue.

* All in all, there will be 3 clues, that are scattered around the community, and You can search and use them anytime to find the solution for provided assignment, after all clues revealed you will have 10 days to earn the badge.

* You can do this individually or as a team (if so let us know who from community is the team).

 

So let’s start. Time is ticking

 

 

 

So the Spel Code limitation was correctly identified by JussiValkonen. In short the problem is how to execute external server side script and get response from it by the means of Spel Code (e.g. execute external program on the server and receive output from it).

 

Assignment is as follows:

 

  • Create a batch file on server with the following content:

 

         @echo off

          echo Output to the spel from batch file succeded!

 

  • Execute a batch file by using your custom Spel Code script and log response in stdlog file from the same Spel Code script.

 

 

 

 

To get you started here is the process on how the solution can be implemented:

 

 

 

You have 5 days before first clue will be revealed.

 

 

First step is to call Perl script from Spel code. So the first clue is hidden in this post:

https://communities.ca.com/message/241850944

Perl script can be executed as usual Spel code, but it should be
done in one particular way, and the clue how to do this is located in this post

https://communities.ca.com/thread/241738128

 

Hopefully you found a way to
execute Perl script and the batch file within that script. The tricky part is
to return output from the batch file to the Spel code. The clue how to do this
is also located in

https://communities.ca.com/thread/241738128

The main trick is that
set_return_data method in Perl is a little bit different than in Spel code. How
different – this You should try and find by yourself.

 

These are all the clues and You have 10 days left to provide us with solution and receive SpelMaster badge.

38 comments
5 views

Permalink

Comments

08-23-2016 05:15 AM

Finaly, i have finished SPEL How to Execute Remote Reference via Perl code document. Here you can find solution for this event.

07-25-2016 11:18 AM

Congratulations JussiValkonen! The ITSM SpelMaster badge has been awarded to you!

07-25-2016 10:57 AM

Congrats on founding the solution. Since i still can not find time to finish SPEL How to Execute Remote Reference via Perl code document, i guess competition is still open. Chris_Hackett do we still have possibility to award badge to JussiValkonen.

07-25-2016 10:34 AM

I returned from summer vacation last week and regardless of the event status on Thursday evening I decided to tackle this. Managed to pull it off and submitted my solution to the given email address and will humbly accept whatever you decide on granting or denying the badge. This was fun, more challenges like this please

06-21-2016 03:27 PM

I have awarded the badges to cdtj and daniel-bighelini . Congratulations!

And...of course I have awarded to you Giedrius!

06-14-2016 04:57 PM

Giedrius,

My script version 2.0 with PERL method.

Thanks by tips.

06-14-2016 04:19 PM

Hi Jussi, i think this depends on how many members is interesting in solving this.

Please anyone interesting in this challenge write comment or drop me an email.

06-14-2016 08:14 AM

You think the duration could be extended a bit? I'm very interested in attempting this but I'm going off-grid today for a month so no chance to try it before coming back

If that's possible then great! If not, then better luck next time I guess

06-09-2016 10:38 PM

I agree Gutis.

Me too dont understand.

This was one of the reasons i publish this vídeo.

06-09-2016 05:02 PM

And CA is developing new interfaces that can not be customized, when they have such flexible solution, that can do magic in good hands. I can not understand that...

Impressive properties Daniel

06-09-2016 04:43 PM

cdtj

About send_catch... In soon i will share more information.

About conditional properties, see this video.

Regards.

06-09-2016 04:34 PM

Hi Daniel,

could you provide little more about send_catch (sorry for offtopic), I played around it today, due to Perl have possibility to call it, I thought that goal could hiding somethere in that methods, but didn't find any clue difference between send_wait.

PS: Properties customization displayed in your video are pretty nice

Regards,

cdtj

06-09-2016 04:02 PM

Thanks @Gutis. I stay happy with your feedback.

I use this solution to consume REST/SOAP services in some webservers in my company and execute a lot of scripts (BATCH, PM, VBS, JAR, etc). Works well.

But I am developing a new version. This new version dont use intermediary temp file. Uses SLUMP API process by send_catch method. But i am having little problems with this. Soon, maybe, i resolve it.

06-09-2016 03:51 PM

I have reviewed your script it is quite impressive.

To be honest this solution is not what i was expecting. Ok to be honest that is something totally unexpected. But i must agree that this is just another approach. I also have learned something new about expand function. I would suggest You to try implement this in my way since this does not require any temporary files loops and manual error control. Chris_Hackett i think we have another badge owner. Congrats daniel-bighelini.

06-09-2016 03:23 PM

This line reads the contents of file and store in variable. Read below in documentation of SPEL:

 

string expand (string format)

Expand variable references in a string.  This is similar to '$' expansion in a shell.  Words in the string to be expanded start with a tag character and are optionally surrounded by {}.  For example $NX_ROOT or ${NX_ROOT}.

 

Five kinds of expansion are possible:

$....    are expanded exactly as shell environment variables.

@....  are expanded as attributes of the "this" pointer (or display  tree data for Wand methods).

&....   are treated as object indirections or translations.  Some key attribute of an object is specified and replaced with the value of some other attribute of the object.

            Syntax: &{val = factory.key->attr} 

The value "val" will be used as a key to identify a Dob of the type specified by "factory".

            If a Dob is uniquely identified, the attribute "attr" will  be fetched and its value will be inserted into the string.

            The value "val" can be a number or a string.  If it begins with a digit and is not quoted, it will be treated as a  number.  Otherwise, it will be treated as a string.  "Val"   strings containg spaces must be quoted.

<....      are treated as a reference to a UNIX file, and replaced with the contents of the file. The file name must be surrounded by braces.  References to environment  variables within braces are not expanded; if a file name includes such references, you must invoke the expand function twice.  See below for an example of this.

 

Single and double quotes are treated as in the shell.  ie. expansions happen within double quotes but not within single quotes. For example:

      "nx root = $NX_ROOT,\n\

      description = @description,\n\

      contact 7 = &{7 = cnt.id->combo_name}"

      file_text = expand("<{" + expand("$HOME/file") + "}")

will expand to

      "nx root = /opt/LGNTpdm

      description = Once upon a time

      contact 7 = Picklemeier, Fred"

      file_text = "the contents of the file"

06-09-2016 03:05 PM

Daniel very interesting but i can not understand how this line reads content of temporary file

zs_file_content = expand("<{" + expand(gsub(zs_filename_bridge_temp, '\\\\', '/')) + "}");

 

Or the temporary file name contains return string

06-09-2016 01:32 PM

Daniel when i will come back home i will review this, it is very exciting it seems like another way arround to achive the same

06-09-2016 01:23 PM

How my solution work?
1) The SPEL function "z_executar" executes the command using the SPEL method "exec" but through an intermediate file controlled by a batch file (z_executar_bridge.cmd);
2) Then in a while loop and using the SPEL method "access" the return file is being awaited by z_executar function;
3) When the return file is generated, they are captured in the return of execution and outgoing messages calling shell;
4) Finally the z_executar function returns the data to the originator of the call in a string containing "ERRORLEVEL @; @ MESSAGE"

 

My source code is free. Enjoy!

06-09-2016 01:14 PM

ENJOY!!!

06-09-2016 01:02 PM

I go do to a video explaining.

please wait a moment.

06-09-2016 12:57 PM

Daniel, i will do that in the mean time could you explain how do you get output from exec command?


06-09-2016 12:41 PM

Gutis,

My function capture return code e return message... Is perfect. Test please... Works!

bop_cmd -f majic\z_executar.spl "z_executar('D:\\test.cmd', 1, 'test')"

 

9508 SIGNIFICANT  bop_cmd.c         381 Starting function z_executar('D:\\test.cmd', 1, 'test') from frg file
9508 SIGNIFICANT  z_functions.spl    54 =============================================================
9508 SIGNIFICANT  z_functions.spl    55 test z_executar -  Inicio : 06/09/2016 13:45:05
9508 SIGNIFICANT  z_functions.spl    56 test z_executar -  Usuario: ''
9508 SIGNIFICANT  z_functions.spl    59 =============================================================
9508 SIGNIFICANT  z_functions.spl  3063 test z_executar -  Quant. de argumentos : 3
9508 SIGNIFICANT  z_functions.spl  3065 test z_executar -  ARG #0 : D:\test.cmd
9508 SIGNIFICANT  z_functions.spl  3065 test z_executar -  ARG #1 : 1
9508 SIGNIFICANT  z_functions.spl  3065 test z_executar -  ARG #2 : test
9508 SIGNIFICANT  z_functions.spl  3081 test z_executar -  Chave de execucao: 1465490705_9785
9508 SIGNIFICANT  z_functions.spl  3094 test z_executar -  Nome do SERVIDOR de execucao: 'SIWDES0471'
9508 SIGNIFICANT  z_functions.spl  3095 test z_executar -  Nome do arquivo de BATCH da BRIDGE: 'D:/PROGRA~1/CA/SERVIC~1/bin/z_executar_bridge.cmd'
9508 SIGNIFICANT  z_functions.spl  3096 test z_executar -  Nome do arquivo de RETORNO da BRIDGE: '\\SIWDES0471\C$\Users\DANIEL~1\AppData\Local\Temp\2\z_executar_bridge_146
9508 SIGNIFICANT  z_functions.spl  3102 test z_executar -  Tentando acessar o arquivo BRIDGE 'D:/PROGRA~1/CA/SERVIC~1/bin/z_executar_bridge.cmd'...
9508 SIGNIFICANT  z_functions.spl  3116 test z_executar -  Executando a linha de comando 'CMD /C D:/PROGRA~1/CA/SERVIC~1/bin/z_executar_bridge.cmd ""D:\test.cmd"" "\\SIWDE
9508 SIGNIFICANT  z_functions.spl  3128 test z_executar -  Tentando acessar o arquivo de RETURN CODE da BRIDGE '\\SIWDES0471\C$\Users\DANIEL~1\AppData\Local\Temp\2\z_execu
9508 SIGNIFICANT  z_functions.spl  3133 test z_executar -  Aguardando 1 segundos antes realizar a tentativa (1/8) de acesso ao arquivo de retorno...
5244 SIGNIFICANT  STDIO               0 09/06/2016 13:45:06,04 DEBUG Definindo local para armazenamento de logs na USD...
5244 SIGNIFICANT  STDIO               0 09/06/2016 13:45:06,07 DEBUG Arquivo de logs utilizado: "D:\PROGRA~1\CA\SERVIC~1\log\z_executar_bridge.log"
5244 SIGNIFICANT  STDIO               0 09/06/2016 13:45:06,08 DEBUG Tamanho maximo do log: 3000000 bytes.
5244 SIGNIFICANT  STDIO               0 09/06/2016 13:45:06,10 DEBUG Verificando tamanho do arquivo de log...
5244 SIGNIFICANT  STDIO               0 09/06/2016 13:45:06,12 DEBUG Tamanho atual do arquivo de log: 8355
5244 SIGNIFICANT  STDIO               0 *************************************************************
5244 SIGNIFICANT  STDIO               0 BAT BRIDGE VIA SPEL 1.03 [z_executar_bridge.cmd]
5244 SIGNIFICANT  STDIO               0 Desenvolvido por Daniel Becker Bighelini em 20/05/2016 10:28
5244 SIGNIFICANT  STDIO               0 Descricao: BRIDGE DE EXECUCAO DE ARQUIVOS VIA SPEL USD
5244 SIGNIFICANT  STDIO               0 *************************************************************
5244 SIGNIFICANT  STDIO               0 09/06/2016 13:45:06,20 DEBUG INICIO
5244 SIGNIFICANT  STDIO               0 09/06/2016 13:45:06,21 DEBUG Capturando variaveis utilizadas pelo SCRIPT...
5244 SIGNIFICANT  STDIO               0 09/06/2016 13:45:06,37 DEBUG Copiando o arquivo temporario "\\SIWDES0471\C$\Users\DANIEL~1\AppData\Local\Temp\2\z_executar_bridge_1
5244 SIGNIFICANT  STDIO               0 1 file(s) copied.
5244 SIGNIFICANT  STDIO               0 09/06/2016 13:45:06,41 DEBUG Excluindo o arquivo temporario "\\SIWDES0471\C$\Users\DANIEL~1\AppData\Local\Temp\2\z_executar_bridge_
5244 SIGNIFICANT  STDIO               0 09/06/2016 13:45:06,43 SIGNIFICANT Operacao CONCLUIDA com sucesso.
5244 SIGNIFICANT  STDIO               0 09/06/2016 13:45:06,45 DEBUG FIM
9508 SIGNIFICANT  z_functions.spl  3133 test z_executar -  Aguardando 2 segundos antes realizar a tentativa (2/8) de acesso ao arquivo de retorno...
9508 SIGNIFICANT  z_functions.spl  3137 test z_executar -  Resultado da tentativa de acesso ao arquivo temporario: 1
9508 SIGNIFICANT  z_functions.spl  3148 test z_executar -  CONTEUDO do arquivo: 'TEST FINISH @ERRORLEVEL@0 '
9508 SIGNIFICANT  z_functions.spl  3151 test z_executar -  Posicao do separador de ERRORLEVEL (@ERRORLEVEL@): 12
9508 SIGNIFICANT  z_functions.spl  3154 test z_executar -  ERRORLEVEL: 0
9508 SIGNIFICANT  z_functions.spl  3156 test z_executar -  RETURN CODE: 'TEST FINISH'
9508 SIGNIFICANT  z_functions.spl  3168 test z_executar -  Execucao concluida com SUCESSO! (0)
9508 SIGNIFICANT  z_functions.spl  3181 test z_executar -  Excluindo arquivo temporario '\\SIWDES0471\C$\Users\DANIEL~1\AppData\Local\Temp\2\z_executar_bridge_1465490705_9
9508 SIGNIFICANT  z_functions.spl    28 =============================================================
9508 SIGNIFICANT  z_functions.spl    29 test z_executar -  Fim: 06/09/2016 13:45:07
9508 SIGNIFICANT  z_functions.spl    30 =============================================================
8476 SIGNIFICANT  connmgr.c        2405 Disconnecting client bop_cmd-#9508

06-09-2016 10:31 AM

Thank you, Gutis for organazing event like this!

It was really fun, as much as programming could be fun

06-09-2016 10:21 AM

Ok guys, it seems we have first badge owner. Congrats cdtj.

I hope that there will be more of you that can solve this till the time will run out.

06-09-2016 10:11 AM

Yes i know, but this is not about executing remote reference with exec() command, it is about executing remote reference and getting real time response from it. This way you can feed some real time information from external system, e.g. status of the user in the AD and etc. On the other hand you have full control on the action, for example you can return an real time error to the SDM user and do not allow to save the object if propagation to external system failed.

06-09-2016 09:52 AM

I have a solution for this.
It consists of two files: one bridge BATCH file and SPEL function control.
To use this z_executar () function, just set up a username and password in the file z_executar_bridge.cmd feeding the variables "batadmin" and "batpassword" and save this file in the $ NX_ROOT \ bin.
The z_executar.spl file must be saved in the $ NX_ROOT \ site \ mods \ majic.
I use this even function to call REST methods through SPEL functions.
z_executar.zip - Google Drive

06-09-2016 03:29 AM

You need to find the way to call perl script, then from perl script execute batch file, get response from it and return this response to your calling spell code

06-09-2016 03:23 AM

Should we solve this by having the Perl script (from your diagram) store output into a temp file then spell read that to then finally log it in stdlog or is the real exercise actually to find out the SDM daemon and method to call the Perl script and get output from that directly to then store in the stdlogs?

 

I guess both are valid options but wanted to have that confirmed.

 

Nice implementation by the way, quite impressive!

06-09-2016 03:02 AM

Thank You cdtj, i hope you will jump in to competition, for you it should be easy, since you are on every clue

06-09-2016 02:57 AM

Thanks for pointing at this great solution. AWESOME!

06-09-2016 02:30 AM

So let's start to search for solution. Post is updated with event assignment and process to solve it

06-09-2016 02:12 AM

Exactly!

06-09-2016 02:06 AM

You needed to figure out how to call arbitrary commands on the server and/or how to get the return values from them?

06-09-2016 01:31 AM

No there is no sound in this video ;)

06-09-2016 01:02 AM

Hi James, Thank You for your feedback!!!

Everything is much simplier, limitation is more generic and possibilities are endless AD is just an example :)

06-09-2016 12:56 AM

Thanks for the great feedback James! Gutis is there supposed to be sound? I didn't hear it either so maybe not

06-09-2016 12:48 AM

Crikey, that is a very impressive demo.  For some reason I couldn't hear the audio track, if there is one - but the result is clearly visible so that hardly mattered.  Taking a punt, it looks to me as though you have worked out how to access a Windows API, exposed as a dll, from spel?  One of the Active Directory interfaces (I couldn't tell which one you used) is exposed as a COM layer, so maybe you've taken that leap as well - but at the very least you've hit the LDAP API.  (Or is that old news that I haven't caught up with yet?)

 

I'm fascinated - and very impressed.  The universe just expanded!

 

Regards,

James

06-08-2016 05:40 PM

Sounds like a great challenge Giedrius!

SpelMaster Event