Automic Workload Automation

  • 1.  Multi Thread CPU for AE

    Posted Jan 08, 2015 10:33 AM
    Currently AE and its processes are only single thread.  The original Automic suggestion was to start up additional DWP and WP processes; however we already have multiple.  They are all only utilizing CPU0 on an 8CPU core server.  This results in high load of CPU0 and less than 1% load of CPU1-7.  We also have multiple CPs, which are also only utilizing CPU0.  After opening a ticket with support, we were advised there is not a way to force AE or its processes to multi thread across cores due to some mysterious supposed issue with java. The java issue was not able to be provided to us in order that we might watch oracle news for when the issue is resolved.  We have been advised an Automic RFE will be opened on our behalf and asked that we also post this to the community here.  See cpu_affinity.png below, noting the PSR (CPU Core Identifier) column.

    j1lojcpqax66.pnghttps://us.v-cdn.net/5019921/uploads/editor/d7/j1lojcpqax66.png" width="912">


    The image below is CPU core utilization broken out by each individual CPU core.  While we are in a happy place at the moment with CPU0 only hovering around 35%, it is not at all difficult to run into a scenario where we are no longer in a happy place.

    2javana1qtl9.pnghttps://us.v-cdn.net/5019921/uploads/editor/lo/2javana1qtl9.png" width="1202">


  • 2.  Multi Thread CPU for AE

    Posted Jan 09, 2015 09:55 PM
    WARNING: This is more of a hack, and I don't know if it actually works.  

    EDIT: It appears there are lots of posts online about Java and multi-core and the application needing to be directly written for  multi-core/multi-threading.  
    http://stackoverflow.com/questions/1223072/forcing-multiple-threads-to-use-multiple-cpus-when-they-are-available

    What if in Task manager you uncheck all but 1 processor for each executable [UCSrvWP.EXE] or [UCSrvCP.EXE]?  
    1. Task manager
    2. Processes Tab (Checkmark or click Show processes for all user)
    3. Sort by Image name
    4. Right-click on UCSrvWP.EXE and chose Set Affinity.
    5. Uncheck all CPU's, except 1 per WP/CP.
    I tried this and when I opened Resource Monitor, it seemed to randomly pop up CPU 1 instead of 0 for me for the WP I selected.  However, it would instantly pop back to 0.

    6t3ilzsvqhxs.png


  • 3.  Multi Thread CPU for AE

    Posted Jan 12, 2015 01:17 PM
    Hi @Jeremy Swartwood,
    Thanks for the suggestion.  Our AE runs on Linux servers.  While there might be some tricking of the processes in Linux at application startup (?), this would not be ideal as it would require additional steps should of our oncall (who may or may not be familiar with the platform) while attempting to recover from a potential outage.  We sort of just expect things these days to be multi thread out of box.  We also would not want to lock any single process to a specific core.  If it requires additional resources, we would want/expect that it is able to.


  • 4.  Multi Thread CPU for AE

    Posted Jan 13, 2015 06:26 AM
    Distributing the load to all CPUs is job of the OS. Checked one of our Linux boxes. Here is the AE processes run on all 4 CPUs.
    Result of ps -aeF
    $ ps -aeF | grep -e "PID" -e "ucsrv[cw]p"
    UID        PID  PPID  C    SZ   RSS PSR STIME TTY          TIME CMD
    root     28633 21906  0 74846 134492  0 06:22 ?        00:00:54 .../ucsrvwp ...
    root     28638 21906  0 75216 134004  0 06:22 ?        00:00:39 .../ucsrvwp ...
    root     28639 21906  0 75190 134952  0 06:22 ?        00:00:39 .../ucsrvwp ...
    root     28640 21906  0 36762 25544   1 06:22 ?        00:00:04 .../ucsrvcp ...
    root     28641 21906  0 36835 25948   1 06:22 ?        00:00:04 .../ucsrvcp ...
    root     28712 21906  0 71930 123248  3 06:30 ?        00:00:47 .../ucsrvwp ...
    root     28713 21906  0 72470 124532  2 06:30 ?        00:00:30 .../ucsrvwp ...
    root     28714 21906  0 72451 124360  1 06:30 ?        00:00:32 .../ucsrvwp ...
    root     28715 21906  0 72793 127088  2 06:30 ?        00:00:36 .../ucsrvwp ...
    root     28716 21906  0 72329 125320  1 06:30 ?        00:00:32 .../ucsrvwp ...
    root     28717 21906  0 37012 26404   3 06:30 ?        00:00:07 .../ucsrvcp ...
    root     28718 21906  0 36730 25404   3 06:30 ?        00:00:06 .../ucsrvcp ...

    I do not much this kind of stuff in Linux. Did you already check if your AE processes run in a cgroup which limits the CPU?$ ps -eo pid,cgroup,psr,args | grep -e "PID" -e "ucsrv[cw]p"
      PID CGROUP   PSR COMMAND
    28633 -          0 .../ucsrvwp ...
    28638 -          0 .../ucsrvwp ...
    28639 -          0 .../ucsrvwp ...
    28640 -          1 .../ucsrvcp ...
    28641 -          1 .../ucsrvcp ...
    28712 -          3 .../ucsrvwp ...
    28713 -          2 .../ucsrvwp ...
    28714 -          1 .../ucsrvwp ...
    28715 -          2 .../ucsrvwp ...
    28716 -          1 .../ucsrvwp ...
    28717 -          3 .../ucsrvcp ...
    28718 -          3 .../ucsrvcp ...

    CGroups are configured in /etc/cgconfig.conf and /etc/cgconfig.d/*.







  • 5.  Multi Thread CPU for AE

    Posted Jan 13, 2015 09:37 AM
    Christian_Bartl_212 
    Thanks.  This is interesting.  Have begun reading through https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/ch01.html

    At first glance, I see this was introduced with Redhat6.  We in fact did recently just upgrade to 6.x from 5.4.  I will post the outcome and any steps required to get there.