Sending logs of on-premise applications to Splunk

  • May 06, 2021

Splunk can read different types of data. After reading the data, it allows search, tag, create reports and dashboards on the data. Mule applications can be configured to send logs to Splunk.

There are two ways to send logs to Splunk:

  • HTTP Appender
  • Splunk HTTP Appender

This article contains steps to send logs from on-premise applications to Splunk using Splunk HTTP appender.

  • Create a token in Splunk
  • Configure log4j2.xml file
  • Add dependencies in pom.xml

Create a token in Splunk

HTTP Event Collector allows you to send data and application events to a Splunk deployment over the HTTP and HTTPS protocols. Using a token you can send data to Splunk.You do not need to include Splunk credentials in your application.

  • Goto splunk UI and select setting > Data inputs.create a new HTTP Event Collector using add new.
  • Provide any name to your token and press next.
  • Then select a Source type that is log4j as we are sending logs to splunk using log4j.
  • Then select all indexes and make Main as Default index.
  • Summary of your token looks something like this :
  • New token is created, which we will use as authorization for splunk.
  • Now go to settings > Data inputs > HTTP Event Collector. You can see the new token created.
  • Go to Global Settings and Enable all Tokens by clicking on Enabled. also add HTTP port as 8088, now your HTTP Event collector will listen on this port.To make HTTP Event collector to listen and communicate over HTTPS rather than HTTP, click the Enable SSL checkbox.
  • If you click on edit you can change any configuration of that token. You can also add a description of that token, name for source .

Configure log4j2.xml file

  • Open src/main/resources/log4j2.xml add the following Splunk HTTP appender to appenders .For sending logs to Splunk from the MuleSoft application, you need to provide an url of splunk for us its “https://localhost:8088” and for Authorization add Token which we have created in splunk.
    <SplunkHttp name=”splunk” url=”https://localhost:8088/”
    token=”7acf9ccb-02a1-482e-a895-ba4ce3934eae” index=”main”
    disableCertificateValidation=”true”>
    <PatternLayout
    pattern=”%-5p %d [%t] [event: %X{correlationId}] %c: %m%n” />
    </SplunkHttp>
  • Then add AppenderRef which is nothing but your SplunkHttp name . in the following image Sends the INFO level logs to splunk and Console.depending on your requirement you can decide which level logs you want to send to splunk.

Add dependencies in pom.xml:

  • Add the following dependencies in your pom.xml dependencies section for Splunk.
    <dependency>
    <groupId>com.splunk.logging</groupId>
    <artifactId>splunk-library-javalogging</artifactId>
    <version>1.7.1</version>
    </dependency>
    <dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-core</artifactId>
    <version>2.10.0</version>
    </dependency>
    <dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-api</artifactId>
    <version>2.10.0</version>
    </dependency>
  • Add following repository in the repositories tag of pom.xml
    <repository>
    <id>splunk-artifactory</id>
    <name>Splunk Releases</name>
    <url>https://splunk.jfrog.io/splunk/ext-releases-local</url>
    </repository>
  • Once all the above configurations are done your application is ready to send logs to splunk.Deploy your application and make a call to it .whatever logs you can see in console are also send to splunk(logs are send depending upon your config in log4j).you can decide based on your requirement which logs you want to print in console and which you want to send to splunk.

Searching logs in Splunk

To see logs go to Splunk > Apps > Search & Reporting:

As you can see in the above image you can either type string to search in the search box or click on data summary.

In the Data Summary there are three options available: hosts, Sources, SourceType. Depending upon your choice you can select one. If we select Sources, you can see it’s the same name that we’ve added to the source of the token we created. Clicking on this will show logs for this source.

You can create separate tokens for different applications so that you can monitor logs separately.

— By Kishori Patil