Redis - OpenTelemetry Collector

Redis is an in-memory data structure that implements a distributed, in-memory key-value database with durability options.

The Sumo Logic app for Redis helps you monitor the Redis database cluster. The preconfigured dashboards provide detailed analysis based on logs. The panels provide details such as RDBMemory Usage, events, RDB, and AOF events.

Redis logs are sent to Sumo Logic through OpenTelemetry filelog receiver.

Log types

This app supports logs for Redis in Cluster mode or Standalone mode. The Redis logs are generated in files as configured in the cluster node configuration files in /etc/redis/redis.conf. For more details on Redis logs, click here.

Creating Fields in Sumo Logic for Redis

The following are Fields that will be created as part of the Redis App install if not already present.

• db.cluster.name. User configured. Enter a name to identify this Redis cluster. This cluster name will be shown in the Sumo Logic dashboards.
• db.system. Has fixed value of redis.
• deployment.environment. User configured. This is the deployment environment where the Redis cluster resides. For example: dev, prod or qa.
• sumo.datasource. Has fixed value of redis.

Prerequisites

This section provides instructions for configuring log collection for Redis running on a non-Kubernetes environment. By default, Redis logs are stored in a log file.

Follow the instructions to set up log collection:

1. To configure the Redis log file, locate your local redis.conf configuration file in the database directory. By default, Redis logs are stored in /var/log/redis/redis-server.log.
2. After determining the location of conf file, open your redis.conf configuration file in a text editor to modify its logging parameters as such:
   1. Specify the server verbosity level. The value loglevel in conf file can be set to one of the following, in ascending order of severity: debug, verbose, notice, warning.
   2. Specify the log file name. Also the empty string can be used to force Redis to log on the standard output. If you use the standard output for logging but daemonize, logs will be sent to /dev/null logfile.
3. Save the redis.conf file and restart the Redis server using the command: sudo service redis-server restart.

Once the logs are configured to write to a local file, follow the steps below to configure the collection in Sumo Logic.

For Linux systems with ACL Support, the otelcol install process should have created the ACL grants necessary for the otelcol system user to access default log locations. You can verify the active ACL grants using the getfacl command. Install the ACL in your Linux environment, if not installed.

The required ACL may not be supported for some rare cases, for example, Linux OS Distro, which is officially not supported by Sumo Logic. In this case, you can run the following command to explicitly grant the permissions.

sudo setfacl -R -m d:u:otelcol-sumo:r-x,d:g:otelcol-sumo:r-x,u:otelcol-sumo:r-x,g:otelcol-sumo:r-x <PATH_TO_LOG_FILE>

Run the above command for all the log files in the directory that need to be ingested, which are not residing in the default location.

If Linux ACL Support is not available, traditional Unix-styled user and group permission must be modified. It should be sufficient to add the otelcol system user to the specific group that has access to the log files.

Collection configuration and app installation

As part of data collection setup and app installation, you can select the App from App Catalog and click on Install App. Follow the steps below.

Step 1: Set up Collector

note

If you want to use an existing OpenTelemetry Collector, you can skip this step by selecting the Use an existing Collector option.

To create a new Collector:

1. Select the Add a new Collector option.
2. Select the platform where you want to install the Sumo Logic OpenTelemetry Collector.

This will generate a command that you can execute in the machine environment you need to monitor. Once executed, it will install the Sumo Logic OpenTelemetry Collector.

Step 2: Configure integration

OpenTelemetry works with a configuration yaml file with all the details concerning the data that needs to be collected. For example, it specifies the location of a log file that is read and sent to the Sumo Logic platform.

In this step, you will configure the yaml file required for Redis Collection.

The log file path configured to capture redis logs must be given here. The files are typically located in /var/log/redis/redis-server.log. If you are using a customized path, check the redis.conf file for this information.

You can add any custom fields which you want to tag along with the data ingested in Sumo. Click on the Download YAML File button to get the yaml file.

Step 3: Send logs to Sumo Logic

Once you have downloaded the yaml file as described in the previous step, follow the below steps based on your platform.

Linux

1. Copy the yaml at /etc/otelcol-sumo/conf.d/ folder in the Redis instance that needs to be monitored.
2. Restart the otelcol-sumo process using:

sudo systemctl restart otelcol-sumo

macOS

1. Copy the yaml at /etc/otelcol-sumo/conf.d/ folder in the Redis instance that needs to be monitored.
2. Restart the otelcol-sumo process using:

otelcol-sumo --config /etc/otelcol-sumo/sumologic.yaml --conf "glob:/etc/otelcol-sumo/conf.d/*.yaml"

Chef

1. Copy the yaml file into your Chef cookbook files directory files/<downloaded_yaml_file>.
2. Use a Chef file resource in a recipe to manage it. cookbook_file '/etc/otelcol-sumo/conf.d/<downloaded_yaml_file>' do mode 0644 notifies :restart, 'service[otelcol-sumo]', :delayed end
3. Add the recipe to your collector setup to start collecting the data. Every team typically has their established way of applying the Chef recipe. The resulting Chef recipe should look something like:

   cookbook_file '/etc/otelcol-sumo/conf.d/<downloaded_yaml_file>' do
       mode 0644
       notifies :restart, 'service[otelcol-sumo]', :delayed
   end

Ansible

1. Place the file into your Ansible playbook files directory.
2. Run the Ansible playbook.

   ansible-playbook -i inventory install_sumologic_otel_collector.yaml
   -e '{"installation_token": "<YOUR_TOKEN>", "collector_tags": {<YOUR_TAGS>}, "src_config_path": "files/conf.d"}'

Puppet

1. Place the file into your Puppet module files directory modules/install_otel_collector/files/<downloaded_yaml>.
2. Use a Puppet file resource to manage it.

   file { '/etc/otelcol-sumo/conf.d/<downloaded_yaml_file>':
       ensure => present,
       source => 'puppet:///modules/install_otel_collector/<downloaded_yaml_file>',
       mode => '0644',
       notify => Service[otelcol-sumo],
     }
3. Apply the Puppet manifest. Every team typically has their established way of applying the Puppet manifest. The resulting Puppet manifest should look something like:

   node 'default' {
       class { 'install_otel_collector'
         installation_token => '<YOUR_TOKEN>',
         collector_tags => { <YOUR_TAGS> },
       }
   
       service { 'otelcol-sumo':
         provider => 'systemd',
         ensure => running,
         enable => true,
         require => Class['install_otel_collector'],
       }
   
       file { '/etc/otelcol-sumo/conf.d/<downloaded_yaml_file>':
         ensure => present,
         source => 'puppet:///modules/install_otel_collector/<downloaded_yaml_file>',
         mode => '0644',
         notify => Service[otelcol-sumo],
       }
     }

After successfully executing the above command, Sumo Logic will start receiving data from your host machine.

Click Next. This will install the app (dashboards and monitors) to your Sumo Logic Org.

Dashboard panels will start to fill automatically. It's important to note that each panel fills with data matching the time range query and received since the panel was created. Results won't immediately be available, but within 20 minutes, you'll see full graphs and maps.

Sample log messages

Here's a sample log message in a non-Kubernetes environment.

5275:S 17 Mar 2021 19:13:38.138 * MASTER <-> REPLICA sync: Finished with success

Sample queries

This sample query is from the Redis - Logs dashboard > Logs panel.

Query string

db.cluster.name=* sumo.datasource="redis"
| json auto maxdepth 1 nodrop
| if  (isEmpty(log), _raw, log)  as message
| count by message
| limit  100
| fields message

Viewing Redis dashboards

Logs

The Redis - Logs dashboard provides a detailed analysis based on logs. The panels provide details such as RDBMemory Usage, events, RDB, and AOF events.

Use this dashboard to:

• Review errors and warnings generated by the server.
• Review the RDBMemory Usage, events, RDB, and AOF events.