Storage

 View Only
Expand all | Collapse all

SPLUNK for VMware

  • 1.  SPLUNK for VMware

    Posted Apr 01, 2009 01:42 PM

    Has anyone had any luck at getting SPLUNK for VMware setup? I'm trying to setup the VMware sourcetypes but I'm obviously doing something wrong configured correctly.

    ________________________________

    Jason D. Langdon



  • 2.  RE: SPLUNK for VMware

    Posted Jun 04, 2009 05:17 PM

    I'm also trying to setup the VMware application for Splunk but can't get it to work. Can anybody provide some problems they ran into and the resolutions to these problems?



  • 3.  RE: SPLUNK for VMware

    Posted Jun 04, 2009 08:19 PM

    I was able to get it up and running for our setup in about 45 min's (forgot to enable the firewall settings)

    Works great

    Eric



  • 4.  RE: SPLUNK for VMware

    Posted Jun 04, 2009 08:41 PM

    I was able to get it up and running for our setup in about 45 min's (forgot to enable the firewall settings)

    I can get the syslog portion working fine but I cannot get the VMware api's to work correctly. I have two guys from SPLUNK look at my log files and config files and neither of them could offer any workable suggestions either.

    ________________________________

    Jason D. Langdon



  • 5.  RE: SPLUNK for VMware

    Posted Jun 04, 2009 08:42 PM

    Where should I be looking for firewall settings? I believe the problem I've run into is that VCenter is expecting a certificate from the Splunk server but for whatever reason the Splunk server isn't sending one. I'm stuck.



  • 6.  RE: SPLUNK for VMware

    Posted Jun 04, 2009 08:59 PM

    Depends....where are you in the setup? Are you installing on Windows? What part are you stuck at?

    Assuming its windows:

    goto a command prompt and type

    echo%JAVAHOME%
    echo %SPLUNK_HOME%

    And paste the results in here...we can start from there.

    Also, please paste in the url from your vmware.conf file (C:\Program Files\Splunk\etc\apps\vmware\default)

    I will try and help you get it up and running.

    Eric



  • 7.  RE: SPLUNK for VMware

    Posted Jun 05, 2009 03:15 PM

    I have Splunk installed on a Linux distribution, Fedora 10. Both of those variables are set, so when I ran the command a blank line was followed. The url from the vmware.conf file is, address of VCenter)/sdk



  • 8.  RE: SPLUNK for VMware

    Posted Jun 05, 2009 03:39 PM

    I dont have a fedora box setup, but I should still be able to help get it going, plus i might build one up.

    More questions:

    1. Can you see the main splunk page when you go through a web browser?

    2. Do you have the VMWare Application already installed?

    3. From the splunk page, restart the splunk service.

    3. Run the command cd $SPLUNK_HOME/etc/apps/vmware

    4. Run the command java -jar lib/splunk.jar

    5. Paste the first 20 lines from step 4.

    Eric



  • 9.  RE: SPLUNK for VMware

    Posted Jun 08, 2009 04:08 PM
    1. yes

    2. yes, it was installed from splunkbase through our splunk interface

    Here is what I grabbed from the test.

    Started

    Caught Exception : Exception : org.apache.axis.AxisFault Message : ; nested exception is:

    gnu.javax.net.ssl.provider.AlertException: UNEXPECTED_MESSAGE: remotely generated; FATAL StackTrace :

    AxisFault

    faultCode: {[http://schemas.xmlsoap.org/soap/envelope/}Server.userException|http://schemas.xmlsoap.org/soap/envelope/%7DServer.userException]

    faultSubcode:

    faultString: gnu.javax.net.ssl.provider.AlertException: UNEXPECTED_MESSAGE: remotely generated; FATAL

    faultActor:

    faultNode:

    faultDetail:

    {[http://xml.apache.org/axis/}stackTrace:gnu.javax.net.ssl.provider.AlertException|http://xml.apache.org/axis/%7DstackTrace:gnu.javax.net.ssl.provider.AlertException]: UNEXPECTED_MESSAGE: remotely generated; FATAL

    at gnu.javax.net.ssl.provider.SSLEngineImpl.unwrap(libgcj.so.9)

    at javax.net.ssl.SSLEngine.unwrap(libgcj.so.9)

    at gnu.javax.net.ssl.provider.SSLSocketImpl.doHandshake(libgcj.so.9)

    at gnu.javax.net.ssl.provider.SSLSocketImpl$SocketOutputStream.write(libgcj.so.9)

    at java.io.BufferedOutputStream.flush(libgcj.so.9)

    at org.apache.axis.transport.http.HTTPSender.writeToSocket(HTTPSender.java:516)

    at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java:138)

    at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)

    at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)

    at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)

    at org.apache.axis.client.AxisClient.invoke(AxisClient.java:165)

    at org.apache.axis.client.Call.invokeEngine(Call.java:2784)

    at org.apache.axis.client.Call.invoke(Call.java:2767)

    at org.apache.axis.client.Call.invoke(Call.java:2443)

    at org.apache.axis.client.Call.invoke(Call.java:2366)

    at org.apache.axis.client.Call.invoke(Call.java:1812)

    at com.vmware.vim.VimBindingStub.retrieveServiceContent(VimBindingStub.java:23449)

    at com.vmware.apputils.vim.ServiceConnection.connect(ServiceConnection.java:54)

    at com.vmware.apputils.vim.ServiceUtil.clientConnect(ServiceUtil.java:36)

    at com.vmware.apputils.AppUtil.connect(AppUtil.java:389)

    at com.splunk.VMWareHostConnection.init(Splunk4VMI.java:275)

    at com.splunk.Splunk4VMI.init(Splunk4VMI.java:393)

    at com.splunk.Splunk4VMI.main(Splunk4VMI.java:573)

    {[http://xml.apache.org/axis/}hostname:APP-07-SPLUNK.gripa.local|http://xml.apache.org/axis/%7Dhostname:APP-07-SPLUNK.gripa.local]

    gnu.javax.net.ssl.provider.AlertException: UNEXPECTED_MESSAGE: remotely generated; FATAL

    at org.apache.axis.AxisFault.makeFault(AxisFault.java:101)

    at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java:154)

    at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)

    at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)

    at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)

    at org.apache.axis.client.AxisClient.invoke(AxisClient.java:165)

    at org.apache.axis.client.Call.invokeEngine(Call.java:2784)

    at org.apache.axis.client.Call.invoke(Call.java:2767)

    at org.apache.axis.client.Call.invoke(Call.java:2443)

    at org.apache.axis.client.Call.invoke(Call.java:2366)

    at org.apache.axis.client.Call.invoke(Call.java:1812)

    at com.vmware.vim.VimBindingStub.retrieveServiceContent(VimBindingStub.java:23449)

    at com.vmware.apputils.vim.ServiceConnection.connect(ServiceConnection.java:54)

    at com.vmware.apputils.vim.ServiceUtil.clientConnect(ServiceUtil.java:36)

    at com.vmware.apputils.AppUtil.connect(AppUtil.java:389)

    at com.splunk.VMWareHostConnection.init(Splunk4VMI.java:275)

    at com.splunk.Splunk4VMI.init(Splunk4VMI.java:393)

    at com.splunk.Splunk4VMI.main(Splunk4VMI.java:573)

    Caused by: gnu.javax.net.ssl.provider.AlertException: UNEXPECTED_MESSAGE: remotely generated; FATAL

    at gnu.javax.net.ssl.provider.SSLEngineImpl.unwrap(libgcj.so.9)

    at javax.net.ssl.SSLEngine.unwrap(libgcj.so.9)

    at gnu.javax.net.ssl.provider.SSLSocketImpl.doHandshake(libgcj.so.9)

    at gnu.javax.net.ssl.provider.SSLSocketImpl$SocketOutputStream.write(libgcj.so.9)

    at java.io.BufferedOutputStream.flush(libgcj.so.9)

    at org.apache.axis.transport.http.HTTPSender.writeToSocket(HTTPSender.java:516)

    at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java:138)

    ...16 more

    Exception running : Splunk4VMI

    Caught Exception : Exception : org.apache.axis.AxisFault Message : ; nested exception is:

    gnu.javax.net.ssl.provider.AlertException: UNEXPECTED_MESSAGE: remotely generated; FATAL StackTrace :

    AxisFault

    faultCode: {[http://schemas.xmlsoap.org/soap/envelope/}Server.userException|http://schemas.xmlsoap.org/soap/envelope/%7DServer.userException]

    faultSubcode:

    faultString: gnu.javax.net.ssl.provider.AlertException: UNEXPECTED_MESSAGE: remotely generated; FATAL

    faultActor:

    faultNode:

    faultDetail:

    {[http://xml.apache.org/axis/}stackTrace:gnu.javax.net.ssl.provider.AlertException|http://xml.apache.org/axis/%7DstackTrace:gnu.javax.net.ssl.provider.AlertException]: UNEXPECTED_MESSAGE: remotely generated; FATAL

    at gnu.javax.net.ssl.provider.SSLEngineImpl.unwrap(libgcj.so.9)

    at javax.net.ssl.SSLEngine.unwrap(libgcj.so.9)

    at gnu.javax.net.ssl.provider.SSLSocketImpl.doHandshake(libgcj.so.9)

    at gnu.javax.net.ssl.provider.SSLSocketImpl$SocketOutputStream.write(libgcj.so.9)

    at java.io.BufferedOutputStream.flush(libgcj.so.9)

    at org.apache.axis.transport.http.HTTPSender.writeToSocket(HTTPSender.java:516)

    at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java:138)

    at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)

    at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)

    at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)

    at org.apache.axis.client.AxisClient.invoke(AxisClient.java:165)

    at org.apache.axis.client.Call.invokeEngine(Call.java:2784)

    at org.apache.axis.client.Call.invoke(Call.java:2767)

    at org.apache.axis.client.Call.invoke(Call.java:2443)

    at org.apache.axis.client.Call.invoke(Call.java:2366)

    at org.apache.axis.client.Call.invoke(Call.java:1812)

    at com.vmware.vim.VimBindingStub.retrieveServiceContent(VimBindingStub.java:23449)

    at com.vmware.apputils.vim.ServiceConnection.connect(ServiceConnection.java:54)

    at com.vmware.apputils.vim.ServiceUtil.clientConnect(ServiceUtil.java:36)

    at com.vmware.apputils.AppUtil.connect(AppUtil.java:389)

    at com.splunk.VMWareHostConnection.init(Splunk4VMI.java:275)

    at com.splunk.Splunk4VMI.init(Splunk4VMI.java:393)

    at com.splunk.Splunk4VMI.main(Splunk4VMI.java:573)

    {[http://xml.apache.org/axis/}hostname:APP-07-SPLUNK.gripa.local|http://xml.apache.org/axis/%7Dhostname:APP-07-SPLUNK.gripa.local]



  • 10.  RE: SPLUNK for VMware

    Posted Jun 08, 2009 04:15 PM

    Looks like you're using the wrong version of java.

    ________________________________

    Jason D. Langdon



  • 11.  RE: SPLUNK for VMware

    Posted Jun 08, 2009 04:56 PM

    I'm using version 1.5, which is supposed to be compatible.



  • 12.  RE: SPLUNK for VMware

    Posted Jun 08, 2009 05:08 PM

    which did you install, java or jdk? I had to download and install jdk1.6.0.13 before it would work.

    ________________________________

    Jason D. Langdon



  • 13.  RE: SPLUNK for VMware

    Posted Jun 08, 2009 07:08 PM

    We upgraded our Java version and installed the matching JDK version. It appears to have fixed the problem. How long does it take for Splunk to index all the data in order for me to see results in the VMware dashboards?



  • 14.  RE: SPLUNK for VMware

    Posted Jun 08, 2009 07:37 PM

    I never did get the VMware dashboards working.

    ________________________________

    Jason D. Langdon



  • 15.  RE: SPLUNK for VMware

    Posted Jun 09, 2009 07:45 PM

    It ran over night and it collected a good deal of data but none of the data was populated for the VMware dashboards. I'll install some ESX updates to see if that solves the problem.



  • 16.  RE: SPLUNK for VMware

    Posted Jun 11, 2009 08:29 PM

    So you're seeing data when you search: sourcetype=vmware_api ?

    If so, whats the latest timestamp you see? Can you click on "Report on Results"? Do you see fields on the left?

    Regarding the dashboards, do you even see them in the pulldown? If not, are you using LDAP or not logging in as "admin"?



  • 17.  RE: SPLUNK for VMware

    Posted Jun 12, 2009 02:23 PM

    All of the saved searches for the VMware app were disabled, so none of the information was being populated in the dashboards. Is this by default? I've enabled all of the saved searches and my dashboards are now being populated.



  • 18.  RE: SPLUNK for VMware

    Posted Jun 08, 2009 06:49 PM

    Hi there - Simon from Splunk here. I would try upgrading to Java 1.6 if you can.

    You can also contact me directly: simon at splunk dot com