Skip to main content
Sumo Logic

Collect Redis Logs and Metrics for Non-Kubernetes environments

We use the Telegraf Operator for Redis metric collection and Sumo Logic Installed Collector for collecting Redis logs. The diagram below illustrates the components of the Redis collection in a non-Kubernetes environment. Telegraf runs on the same system as Redis, and uses the Redis input plugin to obtain Redis metrics, and the Sumo Logic output plugin to send the metrics to Sumo Logic. Logs from Redis on the other hand are sent to either a Sumo Logic Local File or Syslog source. 

clipboard_e7e8945b3dbdd0f2bc864f9777a7e4313.png

This section provides instructions for configuring metrics collection for the Sumo Logic App for Redis. Follow the instructions documented below to set up metrics collection for a given Redis cluster:

  1. Configure Metrics Collection

    1. Configure a Hosted Collector
    2. Configure an HTTP Logs and Metrics Source
    3. Install Telegraf
    4. Configure and start Telegraf
  2. Configure Logs Collection

    1. Configure logging in Redis
    2. Configure Sumo Logic Installed Collector

Step 1 Configure Metrics Collection of Redis Metrics

  1. Configure a Hosted Collector
    To create a new Sumo Logic hosted collector, perform the steps in the Configure a Hosted Collector section of the Sumo Logic documentation.

  2. Configure an HTTP Logs and Metrics Source
    Create a new HTTP Logs and Metrics Source in the hosted collector created above by following these instructions. Make a note of the HTTP Source URL.

  3. Install Telegraf
    Follow the steps in this document Use the in this document following steps to install Telegraf.

  4. Configure and start Telegraf
    As part of collecting metrics data from Telegraf, we will use the Redis input plugin to get data from Telegraf and the Sumo Logic output plugin to send data to Sumo Logic. 

    Create or modify the telegraf.conf file and copy and paste the text below:  

    [[inputs.redis]]
      servers = ["tcp://localhost:6379"]
      namepass = ["redis"]
      fieldpass = ["blocked_clients", "clients", "cluster_enabled", "cmdstat_calls", "connected_slaves", "evicted_keys", "expired_keys", "instantaneous_ops_per_sec", "keyspace_hitrate", "keyspace_hits", "keyspace_misses", "master_repl_offset", "maxmemory", "mem_fragmentation_bytes", "mem_fragmentation_ratio", "rdb_changes_since_last_save", "rejected_connections", "slave_repl_offset", "total_commands_processed", "total_net_input_bytes", "total_net_output_bytes", "tracking_total_keys", "uptime", "used_cpu_sys", "used_cpu_user", "used_memory", "used_memory_overhead", "used_memory_rss", "used_memory_startup"]
      [inputs.redis.tags]
        environment="prod"
        component="database"
        db_system="redis"
        db_cluster="redis_prod_cluster01"
     
    [[outputs.sumologic]]
      url = "<URL Created in Step 3>"
      data_format = "prometheus"
    

Please enter values for the following parameters (marked in bold above):

  • For the input plugins section:
    • servers - The URL to the Redis server. This can be a comma-separated list to connect to multiple Redis servers. Please see this doc for more information on additional parameters for configuring the Redis input plugin for Telegraf.
    • For tags section [inputs.redis.tags]:
      • environment - This is the deployment environment where the Redis cluster identified by the value of servers resides. For example: dev, prod or qa. While this value is optional we highly recommend setting it. 
      • db_cluster - Enter a name to identify this Redis cluster. This cluster name will be shown in the Sumo Logic dashboards. 
  • For output plugins section:
    • url - This is the HTTP source URL created in step 3. Please see this doc for more information on additional parameters for configuring the Sumo Logic Telegraf output plugin.

Here’s an explanation for additional values set by this Telegraf configuration that we request you please do not modify these values as they will cause the Sumo Logic apps to not function correctly.

  • data_format - “prometheus” In the output plugins section, for which Metrics are sent in the Prometheus format to Sumo Logic.
  • db_system: “redis” - In the input plugins section:  This value identifies the database system.
  • component: “database” - In the input plugins section: This value identifies application components.

For all other parameters please see this doc for more properties that can be configured in the Telegraf agent globally.

Once you have finalized your telegraf.conf file, you can start or reload the telegraf service using instructions from their doc.

At this point, Redis metrics should start flowing into Sumo Logic.

Step 2 Configure Collection of Redis Logs

This section provides instructions for configuring log collection for Redis running on a non-Kkubernetes environment for the Sumo Logic App for Redis. 

By default, Redis logs are stored in a log file. Redis also supports forwarding of logs via Syslog.

Sumo Logic supports collecting logs both via Syslog and a local log file. Utilizing Sumo Logic Cloud Syslog will require TCP TLS Port 6514 to be open in your network. Local log files can be collected via Sumo Logic Installed collectors which  requires you to allow outbound traffic to Sumo Logic endpoints for collection to work.

Follow the instructions below to set up log collection:

  1. Configure logging in Redis
  2. Configure local log file or syslog collection
  3. Configure an Installed Collector
  4. Configure a SourceConfigure logging in Redis
Step 1: Configure logging in Redis

Redis supports logging via following methods: syslog, local text log files and stdout. Redis logs have four levels of verbosity. To select a level, set loglevel to one of:

  • debug - This level produces a lot of information, which could be useful in development/testing environments)
  • verbose - This level includes information not often needed, but logs less than debug)
  • notice (moderately verbose, ideal for production environments) - this is the default value
  • warning (only very important / critical messages are logged)

All logging settings are located in redis.conf

Step 2: Configure Redis to log to a local file or syslog

As part of this step, you will make a decision on how to export Redis logs and then accordingly configure your Redis cluster. 

Option 1: Configuring Redis logs to go to log files

With this option, Redis logs written to a log file can be collected via the Local File Source of a Sumo Logic Installed collector.

To configure the Redis log file, locate your local redis.conf configuration file in the database directory. By default, Redis logs are stored in /var/log/redis/redis-server.log.

After determining the location of conf file modify the redis.conf configuration file logging parameters

  1. Open redis.conf configuration file in a text editor.
  2. Set the following config parameters:
    # Specify the server verbosity level.
    # This can be one of:
    # debug (a lot of information, useful for development/testing)
    # verbose (many rarely useful info, but not a mess like the debug level)
    # notice (moderately verbose, what you want in production probably)
    # warning (only very important / critical messages are logged)
    loglevel notice

    # Specify the log file name. Also the empty string can be used to force
    # Redis to log on the standard output. Note that if you use standard
    # output for logging but daemonize, logs will be sent to /dev/null
    logfile ""
  3. Save the redis.conf file and restart the redis server:
    sudo service redis-server restart

Option 2: Configuring Redis logs to stream data to a Syslog source

With this option, Redis logs can be streamed to the Syslog Source of a Sumo Logic Installed Collector.

To configure the Redis syslog, locate your local redis.conf configuration file in the database directory. 

After determining the location of conf file, modify the redis.conf configuration file logging parameters 

  1. Open redis.conf configuration file in a text editor.
  2. Set the following config parameters:
    # To enable logging to the system logger, just set 'syslog-enabled' to yes,
    # and optionally update the other syslog parameters to suit your needs.
    # syslog-enabled no

    # Specify the syslog identity.
    # syslog-ident redis

    # Specify the syslog facility. Must be USER or between LOCAL0-LOCAL7.
    # syslog-facility local0
  3. Save the redis.conf file and restart the redis server:
    sudo service redis-server restart
  4. At this point, Redis will start pushing the logs to the local syslog server.
Step 3: Configuring an Installed Collector

To add an Installed collector, perform the steps as defined on the page Configure an Installed Collector.

Step 4: Configuring a Source

To add a Local File Source source for Redis do the following

  1. Add a Local File Source in the installed collector configured in the previous step.
  2. Configure the Local File Source fields as follows:
  • Name. (Required)
  • Description. (Optional)
  • File Path (Required). Enter the path to your error.log or access.log. The files are typically located in /var/log/redis/redis-server.log. If you are using a customized path, check the redis.conf file for this information. 
  • Source Host. Sumo Logic uses the hostname assigned by the OS unless you enter a different host name
  • Source Category. Enter any string to tag the output collected from this Source, such as Redis/Logs. (The Source Category metadata field is a fundamental building block to organize and label Sources. For details see Best Practices.)
  • Fields. Set the following fields. For more information on fields please see this document:
    1. component = database
    2. db_system = redis
    3. db_cluster = <Your_Redis_Cluster_Name>
    4. environment = <Environment_Name>, such as Dev, QA or Prod.
       

3. Configure the Advanced section: 

  • Enable Timestamp Parsing. Select Extract timestamp information from log file entries.
  • Time Zone. Choose the option, Ignore time zone from log file and instead use, and then select your Redis Server’s time zone.
  • Timestamp Format. The timestamp format is automatically detected.
  • Encoding. Select UTF-8 (Default).
  • Enable Multiline Processing. Detect messages spanning multiple lines
    • Select Infer Boundaries - Detect message boundaries automatically

4. Click Save.