Layer7 API Management

Expand all | Collapse all

Integrating Audit/Logging with Splunk

Jump to Best Answer
  • 1.  Integrating Audit/Logging with Splunk

    Posted 04-10-2018 03:43 PM

    I am attempting to off-box log and audit messages to Splunk using the Splunk Logging Driver for Docker e.g., added this configuration to MGW docker compose yml:

     

    logging:
      driver: splunk
      options:
        splunk-token: "<splunk token>"
        splunk-url: "<splunk url"
        splunk-insecureskipverify: "true"

     

    This method uses the HTTP Event Collector feature of Splunk.  An access token is defined within Splunk and used to gain access to the Splunk HTTP(S) endpoint from the Docker container.

     

    The MGW Docker container builds successfully (connecting to the Splunk instance successfully), but no requests (log messages) are seen on the Splunk instance.  I have tried manipulating the log and audit levels of the MGW, but this has no effect.

     

    Not sure how to resolve this.  Any suggestions are appreciated.



  • 2.  Re: Integrating Audit/Logging with Splunk

    Posted 10-08-2018 09:12 PM

    Hi, 

    I am trying tosimilar solution. Was the splunk solution working for you? I am trying to implement in CA Layer 7 product for Docker image. 



  • 3.  Re: Integrating Audit/Logging with Splunk

    Posted 10-09-2018 05:14 PM

    Hello,

     

    I do not have a Splunk license (since it is third-party) to use in my lab, so I cannot reproduce this. I wanted to ask though... what specific issue or error are you seeing when trying to integrate Splunk with the Container Gateway in Docker? If you can provide some log entries or general behaviour and configuration details, I think this may help us move this forward for you.



  • 4.  Re: Integrating Audit/Logging with Splunk
    Best Answer

    Broadcom Employee
    Posted 12-10-2018 07:17 PM

    We have just setup this in the lab and was able to get it working. We create a new HTTP Event collector:

     

     

    For the docker-compose.yml file we setup the following settings:

     

    version: '3'
    services:
    ssg:
    image: caapim/microgateway:1.0.0-CR01
    extra_hosts:
    - "httpbin.mycompany.com:10.7.36.179"
    - "otk.mycompany.com:10.7.32.187"
    deploy:
    resources:
    limits:
    memory: 2048m
    logging:
    driver: splunk
    options:
    splunk-url: "https://apim-centos7-util.support.local:8088"
    splunk-token: "8da0c475-fbdb-482e-897f-61811bde5ca8"
    splunk-insecureskipverify: "true"
    env_file:
    - ./config/core.env
    - ./config/license-agreement.env
    - ./config/license.env
    - ./config/certificates.env
    - ./config/otk.env
    - ./config/jwt.env
    - ./config/feature-flags.env
    - ./config/solutionkits/policysdk.env
    environment:
    SSG_ADMIN_USERNAME: "admin"
    SSG_ADMIN_PASSWORD: "password"

     

    Finally we built the environment to run using the command:

     

    docker-compose --project-name microgateway                --file docker-compose.yml                --file docker-compose.db.consul.yml                --file docker-compose.lb.dockercloud.yml                up -d --build

     

    In the Splunk search we used the search string of : source="http:Support" (index="history" OR index="main" OR index="summary")

     

     

    Sincerely,

     

    Stephen Hughes

    Broadcom Support



  • 5.  Re: Integrating Audit/Logging with Splunk

    Posted 12-10-2018 09:06 PM

    Thanks for sharing the info Stephen. I already implemented Splunk using the ECS not with docker-compose.yml. Right now our gateway’s routing the sys and audit logs to Splunk.

     

    I am exploring routing the stats to database, it seems this started in 9.4.

     

     

    Thanks & Regards

    Bala



  • 6.  Re: Integrating Audit/Logging with Splunk

    Posted 12-12-2018 12:26 PM

    Yes, in 9.4 we can route Gateway Dashboard statistic data as well to the external db . 



  • 7.  Re: Integrating Audit/Logging with Splunk

    Posted 12-12-2018 02:24 PM

    Hi Prashant,

    Is it possible to move the statistic to external source like Splunk? Or we can only able to move to external db?

     

     

     

     

    Thanks & Regards

     

    Bala



  • 8.  Re: Integrating Audit/Logging with Splunk

    Posted 12-12-2018 11:04 PM

    We can only move statistic data in to  external DB  (This functionality has been added in 9.4 earlier only audit data can be written in to external db)slimier like audit data in to external db  . However you can create dashboard in to splunk for all the data that is getting logged in to traffic log.

    That will cover most of thing that you are seeing over dashboard .



  • 9.  Re: Integrating Audit/Logging with Splunk

    Posted 01-04-2019 05:22 PM

    Hi All,

    I was just calculating the latency the way it handles the logs. Two scenario's I tested. With splunk log-driver the latency is more compare to syslog forwarder. Both scenarios tested on AWS environment.

     

    1. logs --> rsyslog --> configure splunk forwarder

    2. logs --> HTTP Event collector -->Splunk (Splunk log-driver)

    The later one has more latency compare to first one. It might be due to the 1. UDP 2. HTTP.

     

    Does any one observed the same behavior?