Skip to main content
Sumo Logic

Collect Apache Tomcat Logs and Metrics for Non-Kubernetes environments

We use the Telegraf operator for Apache Tomcat metric collection and Sumo Logic Installed Collector for collecting Apache Tomcat logs. The diagram below illustrates the components of the Apache Tomcat collection in a non-Kubernetes environment. Telegraf runs on the same system as Apache Tomcat and uses the Apache Tomcat and Jokokia2 input plugin to obtain Apache Tomcat metrics, and the Sumo Logic output plugin to send the metrics to Sumo Logic. Logs from Apache Tomcat on the other hand are sent to a Sumo Logic Local File source.

This section provides instructions for configuring metrics collection for the Sumo Logic App for Apache Tomcat. Follow the below instructions to set up the metric collection:

  1. Configure Metrics Collection

    1. Configure a Hosted Collector
    2. Configure an HTTP Logs and Metrics Source
    3. Install Telegraf
    4. Download and setup Jolokia on each Apache Tomcat node
    5. Configure and start Telegraf
  2. Configure Logs Collection

    1. Configure logging in Apache Tomcat
    2. Configure Sumo Logic Installed Collector

Step 1 Configure Metrics Collection

  1. Configure a Hosted Collector

To create a new Sumo Logic hosted collector, perform the steps in the Configure a Hosted Collector section of the Sumo Logic documentation.

  1. Configure an HTTP Logs and Metrics Source

Create a new HTTP Logs and Metrics Source in the hosted collector created above by following these instructions. Make a note of the HTTP Source URL.

  1. Install Telegraf

Use the following steps to install Telegraf.

  1. Download and setup Jolokia on each Apache Tomcat node

As part of collecting metrics data from Telegraf, we will use the Jolokia input plugin to get data from Telegraf and the Sumo Logic output plugin to send data to Sumo Logic.

  • Download the latest version of the Jolokia JVM-Agent from Jolokia.
  • Rename downloaded Jar file to jolokia.jar
  • Save the file jolokia.jar on your apache tomcat server in ${TOMCAT_HOME}/webapps
  • Configure Apache Tomcat to use Jolokia
  1. Add following to tomcat-users.xml
    <role rolename="role-CHANGME" />
    <user name="username-CHANGME" password="password-CHANGME" roles="role-CHANGME" />

  2. Start or Restart Apache Tomcat Service

  3. Verify the Jolokia agent installation by curl-ing this URL: http://<tomcat_address>:<tomcat_port>/jolokia/version.

curl -v -u username-CHANGME:password-CHANGME "http://APACHE_TOMCAT_SERVER_IP_ADDRESS:<TOMCAT_PORT>/jolokia/version"

The result looks similar to this

{"request":{"type":"version"},"value":{"agent":"1.6.2","protocol":"7.2","config":{"listenForHttpService":"true","maxCollectionSize":"0","authIgnoreCerts":"false","agentId":"10.0.50.64-6867-1ed563ab-servlet","agentType":"servlet","policyLocation":"classpath:\/jolokia-access.xml","agentContext":"\/jolokia","mimeType":"text\/plain","discoveryEnabled":"false","streaming":"true","historyMaxEntries":"10","allowDnsReverseLookup":"true","maxObjects":"0","debug":"false","serializeException":"false","detectorOptions":"{}","dispatcherClasses":"org.jolokia.http.Jsr160ProxyNotEnabledByDefaultAnymoreDispatcher","maxDepth":"15","authMode":"basic","authMatch":"any","canonicalNaming":"true","allowErrorDetails":"true","realm":"jolokia","includeStackTrace":"true","useRestrictorService":"false","debugMaxEntries":"100"},"info":{"product":"tomcat","vendor":"Apache","version":"7.0.76"}},"timestamp":1625232354,"status":200}
  1. Configure and start Telegraf

As part of collecting metrics data from Telegraf, we will use the Apache Tomcat input plugin and jolokia2 input plugin to get data from Telegraf and the Sumo Logic output plugin to send data to Sumo Logic. 

Create or modify telegraf.conf and copy and paste the text below:  

[[inputs.Tomcat]]
  url = "http://127.0.0.1:8585/manager/status/all?XML=true"
  username = "<username-CHANGEME>"
  password = "<password-CHANGEME>"
  [inputs.tomcat.tags]
    environment="dev_CHANGME"
    component="webserver"
    webserver_system="tomcat"
    webserver_farm="tomcat_on_premise_CHANGEME"
[[inputs.jolokia2_agent]]
  name_prefix = "tomcat_jmx_"
  urls = ["http://localhost:8088/jolokia"]
  username = "username-CHANGME"
  password = "password-CHANGME"
  [inputs.jolokia2_agent.tags]
    environment="dev_CHANGEME"
    component="webserver"
         webserver_system="tomcat"
         webserver_farm="tomcat_on_premise_CHANGEME"
  ### JVM Generic
  [[inputs.jolokia2_agent.metric]]
    name  = "OperatingSystem"
    mbean = "java.lang:type=OperatingSystem"
    paths = ["ProcessCpuLoad","SystemLoadAverage","SystemCpuLoad","TotalPhysicalMemorySize","FreeSwapSpaceSize","TotalSwapSpaceSize","FreePhysicalMemorySize","AvailableProcessors"]
  [[inputs.jolokia2_agent.metric]]
    name  = "jvm_runtime"
    mbean = "java.lang:type=Runtime"
    paths = ["Uptime"]
  [[inputs.jolokia2_agent.metric]]
    name  = "jvm_memory"
    mbean = "java.lang:type=Memory"
    paths = ["HeapMemoryUsage", "NonHeapMemoryUsage", "ObjectPendingFinalizationCount"]

  [[inputs.jolokia2_agent.metric]]
    name     = "jvm_garbage_collector"
    mbean    = "java.lang:name=*,type=GarbageCollector"
    paths    = ["CollectionTime", "CollectionCount"]
    tag_keys = ["name"]
  [[inputs.jolokia2_agent.metric]]
    name       = "jvm_memory_pool"
    mbean      = "java.lang:name=*,type=MemoryPool"
    paths      = ["Usage", "PeakUsage", "CollectionUsage"]
    tag_keys   = ["name"]
    tag_prefix = "pool_"
  [[inputs.jolokia2_agent.metric]]
    name     = "GlobalRequestProcessor"
    mbean    = "Catalina:name=*,type=GlobalRequestProcessor"
    paths    = ["requestCount","bytesReceived","bytesSent","processingTime","errorCount"]
    tag_keys = ["name"]
  [[inputs.jolokia2_agent.metric]]
    name     = "JspMonitor"
    mbean    = "Catalina:J2EEApplication=*,J2EEServer=*,WebModule=*,name=jsp,type=JspMonitor"
    paths    = ["jspReloadCount","jspCount","jspUnloadCount"]
    tag_keys = ["J2EEApplication","J2EEServer","WebModule"]
  [[inputs.jolokia2_agent.metric]]
    name     = "ThreadPool"
    mbean    = "Catalina:name=*,type=ThreadPool"
    paths    = ["maxThreads","currentThreadCount","currentThreadsBusy"]
    tag_keys = ["name"]
  [[inputs.jolokia2_agent.metric]]
    name     = "Servlet"
    mbean    = "Catalina:J2EEApplication=*,J2EEServer=*,WebModule=*,j2eeType=Servlet,name=*"
    paths    = ["processingTime","errorCount","requestCount"]
    tag_keys = ["name","J2EEApplication","J2EEServer","WebModule"]
  [[inputs.jolokia2_agent.metric]]
    name     = "Cache"
    mbean    = "Catalina:context=*,host=*,name=Cache,type=WebResourceRoot"
    paths    = ["hitCount","lookupCount"]
    tag_keys = ["context","host"]
[[outputs.sumologic]]
  url = "<URL Created in Step 3_CHANGEME>”
  data_format = "prometheus"

Please enter values for the following parameters (marked in bold above):

  • In the input plugins section, which is [[inputs.Tomcat]]: 

    • servers - The URL to the Tomcat server. This can be a comma-separated list to connect to multiple Tomcat servers. Please see this doc for more information on additional parameters for configuring the Tomcat input plugin for Telegraf.

    • In the tags section, which is [inputs.Tomcat.tags]

      • environment - This is the deployment environment where the Tomcat farm identified by the value of servers resides. For example: dev, prod or qa. While this value is optional we highly recommend setting it. 

      • Webserver_farm - Enter a name to identify this Tomcat farm. This farm name will be shown in the Sumo Logic dashboards. 

  • In the input plugins section, which is [[inputs.jolokia2_agent]]: 

    • servers - The URL to the Tomcat server. This can be a comma-separated list to connect to multiple Tomcat servers. Please see this doc for more information on additional parameters for configuring the Tomcat input plugin for Telegraf.

    • In the tags section, which is [inputs.jolokia2_agent.tags]:

      • environment - This is the deployment environment where the Tomcat farm identified by the value of servers resides. For example: dev, prod or qa. While this value is optional we highly recommend setting it. 

      • webserver_farm- Enter a name to identify this Tomcat farm. This farm name will be shown in the Sumo Logic dashboards. 

  • In the output plugins section, which is [[outputs.sumologic]]: 

    • url - This is the HTTP source URL created in step 3. Please see this doc for more information on additional parameters for configuring the Sumo Logic Telegraf output plugin.

Here’s an explanation for additional values set by this Telegraf configuration that we request you please do not modify as they will cause the Sumo Logic apps to not function correctly.

  • data_format - “prometheus” In the output plugins section, which is [[outputs.sumologic]]. Metrics are sent in the Prometheus format to Sumo Logic

  • Component: “webserver” - In the input plugins section, which are [[inputs.Tomcat]] and [[inputs.jolokia2_agent]] - This value is used by Sumo Logic apps to identify application components.

  • webserver_system: “tomcat” - In the input plugins sections.In other words, this value identifies the webserver system

  •  For all other parameters please see this doc for more properties that can be configured in the Telegraf agent globally.

Once you have finalized your telegraf.conf file, you can start or reload the telegraf service using instructions from the doc.

At this point, Tomcat metrics should start flowing into Sumo Logic.

Step 2 Configure Logs Collection

This section provides instructions for configuring log collection for Apache Tomcat running on a non-kubernetes environment for the Sumo Logic App for Apache Tomcat. 

By default, Apache Tomcat logs are stored in a log file .Sumo Logic supports collecting logs via a local log file. Local log files can be collected via Installed collectors. An Installed collector will require you to allow outbound traffic to Sumo Logic endpoints for collection to work. For detailed requirements for Installed collectors, see this page.

Based on your infrastructure and networking setup choose one of these methods to collect Tomcat logs and follow the instructions below to set up log collection:

  1. Configure logging in Apache Tomcat

Varnish supports logging via the following methods: local text log files

The Sumo Logic App for Apache Tomcat uses three types of logs

  1. Tomcat Access logs
    Log format description: https://tomcat.apache.org/tomcat-8.0-doc/config/valve.html 
    Recommended pattern used is pattern="common"

  2. Tomcat Catalina.out logs
    Log format description: https://docs.oracle.com/javase/8/docs/api/java/util/logging/SimpleFormatter.html

  3. Tomcat Garbage Collection (GC) logs
    Log format description: https://stackoverflow.com/questions/4468546/explanation-of-tomcat-gc-log-statements

  1. Configure Tomcat to log to a Local file

Configuring Tomcat logs to go to log files

By default, Tomcat logs are stored in /usr/share/tomcat/logs/ The default directory for log files is listed in the /usr/share/tomcat/conf/logging.properties file. Logs from the Tomcat log file can be collected via a Sumo Logic Installed collector and a Local File Source as explained in the next section.

  1. Configuring a Collector

To collect logs directly from the Tomcat machine, configure an Installed Collector.

  1. Configuring a Source

To add a Local File Source source for Apache Tomcat do the following

To collect logs directly from your Tomcat machine, use an Installed Collector and a Local File Source. 

  1. Add a Local File Source.

  2. Configure the Local File Source fields as follows:

  • Name. (Required)
  • Description. (Optional)
  • File Path (Required). Enter the path to your error.log or access.log. The files are typically located in /usr/share/tomcat/logs/*. If you are using a customized path, check the Tomcat.conf file for this information. 
  • Source Host. Sumo Logic uses the hostname assigned by the OS unless you enter a different host name
  • Source Category. Enter any string to tag the output collected from this Source, such as Tomcat/Logs. (The Source Category metadata field is a fundamental building block to organize and label Sources. For details see Best Practices.)

Fields. Set the following fields:

  • component = websystem
  • webserver_system = tomcat
  • webserver_farm = <Your_Tomcat_Farm_Name>
  • environment = <Environment_Name>, such as Dev, QA or Prod.

  1. Configure the Advanced section:

  • Enable Timestamp Parsing. Select Extract timestamp information from log file entries.
  • Time Zone. Choose the option, Ignore time zone from log file and instead use, and then select your Tomcat Server’s time zone.
  • Timestamp Format. The timestamp format is automatically detected.Encoding. Select UTF-8 (Default).
  • Enable Multiline Processing. Detect messages spanning multiple lines
    • Infer Boundaries - Detect message boundaries automatically
  1. Click Save.

At this point, Tomcat logs should start flowing into Sumo Logic.