Skip to main content
Sumo Logic

Collect Couchbase Logs and Metrics for Non-Kubernetes environments

Sumo Logic uses the Telegraf operator for Couchbase metric collection and the Installed Collector for collecting Couchbase logs.

Sumo Logic uses the Telegraf operator for Couchbase metric collection and the Installed Collector for collecting Couchbase logs. The diagram below illustrates the components of the  Couchbase collection in a non-Kubernetes environment. Telegraf uses the Couchbase input plugin to obtain Couchbase metrics and the Sumo Logic output plugin to send the metrics to Sumo Logic. Logs from Couchbase are collected by a Local File Source.

The process to set up collection for Couchbase data is done through the following steps:

  1. Configure Logs Collection
    1. Configure logging in Couchbase
    2. Configure Sumo Logic Installed Collector
    3. Configure a local file source
  2. Configure Metrics Collection
    1. Configure a Hosted Collector
    2. Configure an HTTP Logs and Metrics Source
    3. Install Telegraf
    4. Configure and start Telegraf

Configure Logs Collection

Couchbase app supports the audit log , query log, error log, access log. For details here.

  1. Configure logging in Couchbase.

By default, the Couchbase will write the log to the log directory that was configured during installation. For example, on Linux, the log directory would be /opt/couchbase/var/lib/couchbase/logs. By default, the Audit log is disabled, you must enable the audit log following these instructions. Query log, error log, the access log will be enabled by default.

  1. Configure an Installed Collector. If you have not already done so, install and configure an installed collector for Windows by following the documentation.

  2. Configure a Collector

Use one of the following Sumo Logic Collector options:

  1. To collect logs directly from the Couchbase machine, configure an Installed Collector.

  2. If you are using a service like Fluentd, or you would like to upload your logs manually, configure a Hosted Collector.

  1. Configure a local file source

For an Installed Collector

To collect logs directly from your Couchbase machine, use an Installed Collector and Multi Local File Source.

Repeat steps 1 to 4 for each log source: audit log, query log, error log, access log. 

  1. Add a Local File Source.

  2. Configure the Local File Source fields as follows:

  • Name. (Required)

  • Description. (Optional)

  • File Path (Required). Enter the path to your log files: The files are typically located in folder /opt/couchbase/var/lib/couchbase/logs. 

    • For Audit Log: /opt/couchbase/var/lib/couchbase/logs/audit.log

    • For Error Log: /opt/couchbase/var/lib/couchbase/logs/error.log

    • For Access Log: /opt/couchbase/var/lib/couchbase/logs/http_access.log

    • For Query Log: /opt/couchbase/var/lib/couchbase/logs/query.log

  • Source Host. Sumo Logic uses the hostname assigned by the OS unless you enter a different hostname.

  • Source Category. Enter any string to tag the output collected from this Source, such as Couchbase/AccessLog for access log. (The Source Category metadata field is a fundamental building block to organize and label Sources. For details see Best Practices.)

  • Fields. Set the following fields
    component = database
    db_system = couchbase
    db_cluster = <Your_Couchbase_Cluster_Name>. Enter Default if you do not have one.
    environment = <Your_Environment_Name> (for example, Dev, QA, or Prod).

  1. Configure the Advanced section:

  • Enable Timestamp Parsing. Select Extract timestamp information from log file entries.

  • Time Zone. Automatically detect.

  • Timestamp Format. The timestamp format is automatically detected.

  • Encoding. Select UTF-8 (Default).

  • Enable Multiline Processing

    • Error logs. Select Detect messages spanning multiple lines and Infer Boundaries - Detect message boundaries automatically.

    • Access logs. These are single-line logs, uncheck Detect messages spanning multiple lines.

  1. Click Save.

For a Hosted Collector

If you are using a service like Fluentd, or you would like to upload your logs manually, use a Hosted Collector and an HTTP Source.

  1. Add an HTTP Source.

  2. Configure the HTTP Source fields as follows:

  • Name. (Required)

  • Description. (Optional)

  • Source Host. Sumo Logic uses the hostname assigned by the OS unless you enter a different hostname.

  • Source Category. Enter any string to tag the output collected from this Source, such as Couchbase/AccessLog for access log. (The Source Category metadata field is a fundamental building block to organize and label Sources. For details see Best Practices.)

  1. Configure the Advanced section:

  • Enable Timestamp Parsing. Select Extract timestamp information from log file entries.

  • Time Zone. For Access logs, use the time zone from the log file. For Error logs, make sure to select the correct time zone.

  • Timestamp Format. The timestamp format is automatically detected.

  • Enable Multiline Processing

    • Error logs: Select Detect messages spanning multiple lines and Infer Boundaries - Detect message boundaries automatically.

    • Access logs: These are single-line logs, uncheck Detect messages spanning multiple lines.

  1. Click Save.

  2. When the URL associated with the HTTP Source is displayed, copy the URL so you can add it to the service you are using, such as Fluentd.

Configure Metrics Collection

Setup a Sumo Logic HTTP Source

  1. Configure a Hosted Collector for Metrics.
    To create a new Sumo Logic hosted collector, perform the steps in the Configure a Hosted Collector documentation.

  2. Configure an HTTP Logs & Metrics source:

    1. On the created Hosted Collector on the Collection Management screen, select Add Source.

    2. Select HTTP Logs & Metrics.

      1. Name. (Required). Enter a name for the source.

      2. Description. (Optional).

      3. Source Category (Recommended). Be sure to follow the Best Practices for Source Categories. A recommended Source Category may be Prod/DBServer/Couchbase/Metrics.

    3. Select Save.

    4. Take note of the URL provided once you click Save. You can retrieve it again by selecting the Show URL next to the source on the Collection Management screen.

Setup Telegraf

  1. Install Telegraf if you haven’t already. Use the following steps to install Telegraf.

  2. Configure and start Telegraf.
    As part of collecting metrics data from Telegraf, we will use the Couchbase input plugin to get data from Telegraf and the Sumo Logic output plugin to send data to Sumo Logic.

    Create or modify telegraf.conf and copy and paste the text below: 

  [[inputs.couchbase]]
servers = ["http://<USER_TO_BE_CHANGED>:<PASS_TO_BE_CHANGED>@localhost:8091"]
bucket_stats_included = ["*"]
[inputs.couchbase.tags]
   db_cluster="<ClusterName_TO_BE_CHANGED>"
   component="database"
   environment="<env_TO_BE_CHANGED>"
   db_system="couchbase"


[[outputs.sumologic]]
  url = "<URL_from_HTTP_Logs_and_Metrics_Source>"
  data_format = "prometheus"

Enter values for fields annotated with <VALUE_TO_BE_CHANGED> to the appropriate values. Do not include the brackets (<>) in your final configuration

  • Input plugins section, which is [[inputs.couchbase]]:

    • servers: This is the endpoint of the management portal of couchbase server. For details, see this doc .

  • In the tags section, which is [inputs.couchbasesnmp.tags]:

  • environment - This is the deployment environment where the Couchbase server identified by the value of servers resides. For example; dev, prod, or QA. While this value is optional we highly recommend setting it. 

  • db_cluster - Enter a name to identify this Couchbase cluster. This cluster name will be shown in our dashboards. 

  • In the output plugins section, which is [[outputs.sumologic]]

    • URL - This is the HTTP source URL created previously. See this doc for more information on additional parameters for configuring the Sumo Logic Telegraf output plugin.

Here’s an explanation for additional values set by this Telegraf configuration.

  • data_format: “prometheus” - In the output [[outputs.sumologic]] plugins section. Metrics are sent in the Prometheus format to Sumo Logic.
  • component - “database” - In the input [[inputs.couchbase]] plugins section. This value is used by Sumo Logic apps to identify application components.
  • db_system - “couchbase” - In the input plugins sections. This value identifies the database system.

See this doc for all other parameters that can be configured in the Telegraf agent globally.

At this point, Telegraf should start collecting the Couchbase metrics and forward them to the Sumo Logic HTTP Source.