Release Automation

 View Only
Expand all | Collapse all

Arbitrary Number of Multiple Simultaneous Actions

  • 1.  Arbitrary Number of Multiple Simultaneous Actions

    Posted Feb 22, 2016 06:27 PM

    Hello all.

     

    I have the following:

     

    • A simple flow that contains 2 sub-flows.
    • Each sub-glow contains 1 action.
    • The second sub-flow loops through a list of ip addresses and runs a command line action for each address.

     

    Instead of looping through the list of ip addresses (from an string array environment variable), I would like the second sub-flow to kick off the command line action for each of the ip addresses at the same time (more-or-less). In other words I want to spawn a number of command line processes simultaneously and monitor each one for success/failure. The number of ip addresses for which this would be done will vary from environment to environment.

     

    I cannot find any existing actions that would facilitate this. Does anybody know of a way to accomplish what I'm trying to do without getting into the SDK?

     

    Thanks in advance for any feedback.



  • 2.  Re: Arbitrary Number of Multiple Simultaneous Actions

    Posted Feb 23, 2016 01:20 AM

    you will need a loop somewhere, at least that's my guess, either you have it within the flow or you need a batch/powershell/javascript where you loop through it by accepting for example a comma separated list in your call.

     

    on the other hand I might not really get what you try to accomplish. How is your current set up? within the loop are you only able to really just monitor one IP, which means, you need to wait until the monitoring is over before the next one gets it go or are you just starting the monitoring and then jump to the next ip. The last one needs to be like this wherever you do it, unless your using threads and even these need to be created in a sequential manner first.



  • 3.  Re: Arbitrary Number of Multiple Simultaneous Actions

    Posted Feb 23, 2016 09:34 AM

    Thanks Michael.

     

    I already have a loop on the second subflow. It loops through the list of ip addresses. Be default the CLI action executes for one of the ip addresses, (each being an array element), at a time and only continues to the next if the current execution is successful. There is an option on the CLI action that executes the command as a background process. That results in the command for each ip address to be run in rapid succession - almost simultaneous. However, once they are kicked off, they are not monitored for success or failure which is not really useful for me.

     

    I think, ideally, having an action to spawned as many simultaneous "virtual" actions at runtime as needed, or allowed by some configurable constraint, would really suit my needs. I understand that CA Release Automation does not have a concept of a "virtual action." I don't know if it's possible to accomplish something like that using the SDK. I do have the option of modifiying the application that RA is kicking off with the CLI action to solve this problem there, but had hoped there might be a way to have RA do it.



  • 4.  Re: Arbitrary Number of Multiple Simultaneous Actions
    Best Answer

    Posted Feb 23, 2016 09:41 AM

    well, just a quick brainstorming kinda idea, which I assume you might not really like

     

    it might be possible to run this stuff in parallel, when setting up a subdeployment, where you have an ip adress check per deployment step.

     

    so basically your deployment triggers a new deployment which only checks the IPs, well of course the agent you would use for this subdeployments needs to have parallel execution enabled.

     

    as I said, don't think you like it, as it seems even for me a bit over the top for the request, but it might work



  • 5.  Re: Arbitrary Number of Multiple Simultaneous Actions

    Posted Feb 23, 2016 09:47 AM

    Interesting idea. I'll look into it further.

    If it works out, I'll make sure to flag your description as the correct answer



  • 6.  Re: Arbitrary Number of Multiple Simultaneous Actions

    Posted Feb 23, 2016 09:58 AM

    alright, looking forward to your results, if you've tried it :-)



  • 7.  Re: Arbitrary Number of Multiple Simultaneous Actions

    Posted Feb 23, 2016 03:15 PM

    Hi Micheal. I think this can work, (though I'm not sure yet if we would go with it long term.) I'm trying to use the "ROC - Run Deployment plan" action to generate a deployment plan and deployment from a template and run it, but it requires not only the Template name and Application name, but also the IDs for both of those objects. I cannot understand why it needs both. A quick reference of the API indicates only 1 of each should be needed. Do you know how I can find the Template ID and Application ID I need? I don't see any actions that I can use for that either.

     

    Any help you can offer is greatly appreciated.

     

    FTR, I had been pursuing the use of "ROC - Create Deployment..." and "ROC - Run Deployment" actions. I was not able to get the "ROC - Create Deployment From Template Category" to work at all. It always returns the following error:

     

    (#1) Failed to create deployment for application [AppName] , template category [test_tmpl_category] , environment [Dev].

     

    Internal Error:

    HTTP error 500: Internal Server Error

    Unexpected error on server [prepare stage is null.].

     

    My guess with that is that the action takes inputs of Application Name, Environment Name, Template Category Name, Deployment Name, Deployment Version, etc. It does not take an input of Template Name. Since there can be multiple templates in a template category, I'm guessing that the action will always fail because it never has a value for Template Name.



  • 8.  Re: Arbitrary Number of Multiple Simultaneous Actions

    Posted Feb 24, 2016 01:42 AM

    not sure if I can help you with the trouble you're having, as I would need to look into the flow and further more I'm only a small designer my self :-D

     

    but I can tell you how we did set up something similar, maybe this can point you in some better directions.

     

    let me quickly explain what our goal was for this set up:

    one of the projects we're deploying has several subprojects and the coordination of the deployments takes a lot of effort, so we wanted something like a master on top of it, so that one deployment takes care of the deployment of all subprojects.

     

    to achieve this we do the following (I'm only pointing out the steps to create and run the subdeployments)

    1. Predeployment of the master
      1. REST call to "create-deployment-plan", which is better as the action, as you can enter the application name and template name instead of the ID in the body of the REST call
      2. From the REST call output we get the deployment ID with the RegEx: deploymentPlanId":"(\d+) and saving it in a parameter for the later usage
      3. Then we just add the the Subdeployment as a step to the master with ROC - Create Deployment Step
    2. Running the master
      1. Again we're using a REST call, this time to "run-deployments" and in the body for this we have the application name (built-in param) the plan ID and deployment name, which we both saved in the predeployment, and the environment, which comes from the built-in parameter
      2. For some monitoring we're grabbing the deployment id from the REST output with the RegEx: with id \[(\d+)\]

     

    as I said don't know if this really helps, but I think the least thing you can take from it is to try using the REST calls instead of the ROC actions.



  • 9.  Re: Arbitrary Number of Multiple Simultaneous Actions

    Posted Feb 24, 2016 08:38 AM

    Thanks for the insights Micheal. For the time being I've made progress with the "ROC - Run Deployment Plan" action by getting the IDs I need using REST calls. Now I need to pass some deployment specific data from the master to the sub-deployment. The specific data relates to an artifact, so I think I can just create/assign an artifact package to each sub-deployment and pull the data out of the artifact.

     

    In scenario you describe, does the master deployment finish once it's kicked off it's sub-deployments, or does it wait until they are all successful before completing? I'm thinking of using an action (REST?) to gets statuses for the sub-deployments to keep the master alive until everything completes.



  • 10.  Re: Arbitrary Number of Multiple Simultaneous Actions

    Posted Feb 24, 2016 08:42 AM

    yep, that is what we're doing, the master monitors the sub-deployment and finishes after all sub-deployments are done



  • 11.  Re: Arbitrary Number of Multiple Simultaneous Actions

    Posted Feb 26, 2016 05:30 AM

    Kicking off sub-deployments as described above is a work around as far as I am concerned and provides for a lot of overhead.

    I have this similar use case, where we would like to run actions inside a flow in parallel.

    So I suggest to make this into a feature request:

    Give an addition option to the flows, where next to the diferent loops you can also select "parallel".

    This "loop"-type also expects an array and will output each array item in a parameter, for use inside the flow. (like the foreach loop)

    Optionally you could provide a "Number of threads" integer, which the flow uses to limit the maximum of parallel executions.



  • 12.  Re: Arbitrary Number of Multiple Simultaneous Actions

    Posted Feb 29, 2016 11:30 AM

    Robert, I agree with your assessment that the sub-deployment approach is a work-around with a lot of overhead. Additionally, I've had a quite a bit more overhead to add in the the "Number of threads" type of functionality using nested loops and multiple arrays. I'm debugging that now.

     

    I'm new to the "CA Communities" and don't know how to submit a feature request other than by direct contact with a CA person. Doing that doesn't provide the opportunity for folks like yourself to add your support and comments. Can you point me in the right direction to submitting the feature request via the Communities pages? Thanks.



  • 13.  Re: Arbitrary Number of Multiple Simultaneous Actions

    Posted Mar 01, 2016 04:30 AM

    when you're on the community start page you will find "actions" in the navigation, there you can select "idea"

     

    01-03-2016 10-29-03.png