Skip to main content
Sumo Logic

Collect Logs and Metrics for Redis ULM

This page provides instructions for collecting logs for the Sumo App for Redis ULM.

This page provides instructions for configuring log and metric collection for the Sumo Logic App for Redis.

Collection Process Overview

Configuring log and metric collection for the Redis ULM App includes the following tasks:

Collect Logs and Metrics for Redis ULM

This page provides instructions for configuring log and metric collection for the Sumo Logic App for Redis.

Collection Process Overview

Configuring log and metric collection for the Redis ULM App includes the following tasks:

  • Step 1: Collect Logs for Redis

    • Non-Kubernetes.

    • Kubernetes,

  • Step 2: Collect Metrics for Redis

    • Non-Kubernetes

    • Kubernetes

Configure logging in Redis ULM

All logging settings are located in /etc/redis/redis.conf. Redis supports logging via two methods: syslog and local text log files. Redis logs have four levels of verbosity. To select a level, set loglevel to one of:

  • debug (a lot of information, useful for development/testing)
  • verbose (includes information not often needed, but logs less than debug)
  • notice (moderately verbose, ideal for production environments) - default
  • warning (only very important / critical messages are logged)

Local Log File 

By default, Redis logs are stored in /var/log/redis/redis-server.log.  The default directory can be updated in redis.conf. 

If you would like to use local log files for logging, utilize Sumo Logic Installed collector or tools like fluentd to send logs to Sumo Logic.

Syslog

To send Redis logs to Sumo Logic using the host’s native syslog, configure a syslog tool such as syslog-ng or rsyslog, then enable syslog within Redis by uncommenting the syslog-enabled line and setting it to yes in redis.conf.

The syslog identity tag can be overridden from the default of Redis by uncommenting and updating syslog-ident variable.

Non-Kubernetes

This section provides instructions for configuring log collection for Redis running on a non-kubernetes environment for the Sumo Logic App for Redis ULM. Follow the below instructions to set up the Log collection.

  1. Configure a Collector
  2. Configure a Source
Configuring a Collector

Use one of the following Sumo Logic Collector options:

  • To collect logs directly from the Redis machine, configure an Installed Collector.
  • If you are using a service like Fluentd or Syslog, or you would like to upload your logs manually, configure a Hosted Collector
Configure a Source

For an Installed Collector

To collect logs directly from your Redis ULM machine, use an Installed Collector and a Local File Source. 

  1. Add a Local File Source.
  2. Configure the Local File Source fields as follows:
  • Name. (Required)
  • Description. (Optional)
  • File Path (Required). Enter the path to your error.log or access.log. The files are typically located in /var/log/redis/redis-server.log. If you are using a customized path, check the redis.conf file for this information. 
  • Source Host. Sumo Logic uses the hostname assigned by the OS unless you enter a different host name
  • Source Category. Enter any string to tag the output collected from this Source, such as Redis/Logs. (The Source Category metadata field is a fundamental building block to organize and label Sources. For details see Best Practices.)
  1. Configure the Advanced section:

  • Enable Timestamp Parsing. Select Extract timestamp information from log file entries.
  • Time Zone. Choose the option, Ignore time zone from log file and instead use, and then select your Redis Server’s time zone.
  • Timestamp Format. The timestamp format is automatically detected.
  • Encoding. Select UTF-8 (Default).
  • Enable Multiline Processing. Detect messages spanning multiple lines
    • Infer Boundaries - Detect message boundaries automatically
  1. Click Save.

For a Hosted Collector

If you are using a service like Fluentd, or you would like to upload your logs manually, use a Hosted Collector and an HTTP Source.

  1. Add an HTTP Source.
  2. Configure the HTTP Source fields as follows:
    • Name. (Required)
    • Description. (Optional)
    • Source Host. Sumo Logic uses the hostname assigned by the OS unless you enter a different host name.
    • Source Category. Enter any string to tag the output collected from this Source, such as Redis/logs. (The Source Category metadata field is a fundamental building block to organize and label Sources. For details see Best Practices.)
  3. Configure the Advanced section:
    • Enable Timestamp Parsing. Select Extract timestamp information from log file entries.
    • Time Zone. Choose the option, Ignore time zone from log file and instead use, and then select your Redis Server’s time zone.
    • Timestamp Format. The timestamp format is automatically detected.
    • Encoding. Select UTF-8 (Default).
    • Enable Multiline Processing. Detect messages spanning multiple lines
      • Infer Boundaries - Detect message boundaries automatically
  4. Click Save.
  5. When the URL associated with the HTTP Source is displayed, copy the URL so you can add it to the service you are using, such as Fluentd.

For Syslog

If you are using Syslog, use a Hosted Collector and a Cloud Syslog Source.

  1. Add a Cloud Syslog Source.
  2. Configure the HTTP Source fields as follows:
    • Name. (Required)
    • Description. (Optional)
    • Source Host. Sumo Logic uses the hostname assigned by the OS unless you enter a different host name.
    • Source Category. Enter any string to tag the output collected from this Source, such as Redis/logs. (The Source Category metadata field is a fundamental building block to organize and label Sources. For details see Best Practices.)
  3. Configure the Advanced section:
    • Enable Timestamp Parsing. Select Extract timestamp information from log file entries.
    • Time Zone. Choose the option, Ignore time zone from log file and instead use, and then select your Redis Server’s time zone.
    • Timestamp Format. The timestamp format is automatically detected.
    • Encoding. Select UTF-8 (Default).
    • Enable Multiline Processing. Detect messages spanning multiple lines
      • Infer Boundaries - Detect message boundaries automatically
  4. Click Save.
  5. When the URL, Port, and Host associated with the Cloud Syslog Source is displayed, copy the values so you can add it to the Syslog service you are using.
  6. Configure a Syslog tool of your choice on Redis server such as syslog-ng or rsyslog, copy the URL, Port and Host generated in the previous step.
Kubernetes

Use the Sumologic-Kubernetes-Collection, to send the logs to Sumologic. For more information, visit.

Step 2: Collect Metrics for Redis ULM

Non-Kubernetes

This section provides instructions for configuring metrics collection for the Sumo Logic App for Redis. Follow the below instructions to set up the metric collection.

  1. Configure Metrics in Redis ULM
  2. Configure a Hosted Collector
  3. Configure a Http Logs and Metrics Source
  4. Install Telegraf
  5. Configure Telegraf and Forward Metrics to Sumo Logic
Configure Metrics in Redis ULM

This section provides instructions for configuring metrics collection for the Sumo Logic App for Redis. Follow the below instructions to set up the metric collection.

  1. Configure a Hosted Collector
  2. Configure an Http Logs and Metrics Source
  3. Install Telegraf
  4. Configure and start Telegraf
Configure a Hosted Collector

To create a new Sumo Logic hosted collector, perform the steps in the Configure a Hosted Collector section of the Sumo Logic documentation.

Configure an Http Logs and Metrics Source

Create a new HTTP Logs and Metrics Source in the hosted collector created above by following these instructions. 

  • Make a note of HTTP Source URL.

Use the following steps to install Telegraf.

Configure and start Telegraf

Create a file called telegraf.conf and add the appropriate configuration. The following is a basic example:

[agent]
  interval = "60s"
 
[[inputs.redis]]
  servers = ["tcp://localhost:6379"]
  namepass = ["redis"]
  fieldpass = ["blocked_clients", "clients", "cluster_enabled", "cmdstat_calls", "connected_slaves", "evicted_keys", "expired_keys", "instantaneous_ops_per_sec", "keyspace_hitrate", "keyspace_hits", "keyspace_misses", "master_repl_offset", "maxmemory", "mem_fragmentation_bytes", "mem_fragmentation_ratio", "rdb_changes_since_last_save", "rejected_connections", "slave_repl_offset", "total_commands_processed", "total_net_input_bytes", "total_net_output_bytes", "tracking_total_keys", "uptime", "used_cpu_sys", "used_cpu_user", "used_memory", "used_memory_overhead", "used_memory_rss", "used_memory_startup"]
 
[[outputs.sumologic]]
  url = "<URL Created in Step 3>"
  data_format = "prometheus"
  • interval - This is the frequency to send data to Sumo Logic, in this example, we will send the metrics every 60 seconds. Please refer to this doc for more properties that can be configured in the Telegraf agent globally.
  • servers - The URL to the Redis server. This can be a comma-separated list to connect to multiple Redis servers. Please refer to this doc for more information on configuring the Redis input plugin for Telegraf.
  • url - This is the HTTP source URL created in step 3. Please refer to this doc for more information on configuring the Sumo Logic Telegraf output plugin.
  • data_format. The format to use when sending data to Sumo Logic. Please refer to this doc for more information on configuring the Sumo Logic Telegraf output plugin.

Once you have finalized your telegraf.conf file, you can run the following command to start telegraf.

telegraf --config /path/to/telegraf.conf

Kubernetes

The following steps assume you are collecting Redis metrics from a Kubernetes environment. In a Kubernetes environment, we use the Telegraf Operator, which is packaged with our Kubernetes collection. You can learn more on this here.

  1. Set up Kubernetes Collection with the Telegraf Operator.
  2. On your Redis Pods, add the following annotations to configure Telegraf.
annotations:
        telegraf.influxdata.com/inputs: |+
          [[inputs.redis]]
            servers = ["tcp://localhost:6379"]
        telegraf.influxdata.com/class: sumologic-prometheus
        prometheus.io/scrape: "true"
        prometheus.io/port: "9273"
  • telegraf.influxdata.com/inputs - This contains the required configuration for the Telegraf Redis Input plugin. Please refer to this doc for more information on configuring the Redis input plugin for Telegraf. Note: As telegraf will be run as a sidecar the host should always be localhost.
  • telegraf.influxdata.com/class: sumologic-prometheus - This instructs the Telegraf operator what output to use. This should not be changed.
  • prometheus.io/scrape: "true" - This ensures our Prometheus will scrape the metrics.
  • prometheus.io/port: "9273" - This tells prometheus what ports to scrape on. This should not be changed.

Sample Log Messages

Query Sample

This sample Query is from the Master <--> Replica Sync Events panel of the Redis ULM - Logs dashboard.

Query String

_sourceCategory="Labs/redislogs"
| parse "MASTER <-> REPLICA sync: *" as sync_event
| replace( sync_event, /(receiving )(.*? .*?)( from master to disk)/, "receiving data from master to disk") as sync_event
| count by sync_event