Automic Workload Automation

Expand all | Collapse all

Hadoop Agent Installation

  • 1.  Hadoop Agent Installation

    Posted 08-15-2017 09:31 AM
    Hi,

    Does Hadoop agents works on Application server or do I have to put the agents on Hadoop servers?
    Is there way to execute IMPALA jobs on Automic?


  • 2.  Hadoop Agent Installation

    Posted 08-15-2017 10:43 AM
    Oguz,
    Depends.

    The simplest way would be to place an agent on one or more of the Hadoop servers.   
    If Hadoop automatically distributes load across nodes, this is likely to be sufficient.  
    If Hadoop can be invoked via a web service call, then no need for a Hadoop 'local' OS agent.   You will need the UC4 WebService agent on some device.
    You will need to test exactly how Hadoop presents status to UC4.

    One that provides limited capabilities and Automic will not suggest - if the Hadoop server(s)  will allow remote script submission at the OS level....   You will need to test this to see EXACTLY how the submission is done and if the connection is maintained until the submission is complete OR until the launched process is complete




  • 3.  Hadoop Agent Installation

    Posted 08-15-2017 04:56 PM
    Hi Oguz_Mutlu_775,

    You can install the RA Hadoop agent remote. There's no need for it to run on the application server(s) itself.
    Right now the agent supports the following job types:

    HDFS
    HIVE
    MAPREDUCEJAR
    MAPREDUCESTREAM
    PIG

    IMPALA is not part of it right now. You might want to create a product enhancement for this feature to be implemented in a future release.

    Cheers,
    Alexander



  • 4.  Hadoop Agent Installation

    Posted 08-15-2017 05:00 PM