Skip to main content
Sumo Logic

Collect Logs from Docker

This procedure documents how to collect events and statistics from Docker into Sumo Logic. The Sumo Logic App for Docker provides operational insight into your Docker containers. The App includes Dashboards that allow you to view your container performance statistics for CPU, memory, and the network. It also provides visibility into container events such as start, stop, and other important commands.

You can add the following types of Docker Sources to an Installed Collector:

  • Docker Logs. Collects stdout/stderr logs from processes that are running within Docker containers.
  • Docker Stats. Collects metrics about Docker containers.

For more information about how Sumo Logic collects information from Docker, see Docker Sources. Or, refer to the Docker API documentation.

Log Types

The Sumo Logic App for Docker gathers statistics and events from the Docker Remote API on each host.

For more information about Docker events, see Monitor Docker Events

For information about Docker statistics based on resource usage, see Docker Container Stats.

Prerequisites/Requirements

The Sumo Logic collector uses the Docker Remote API to collect Docker logs. The log driver configured on the Docker container must use either the json-file or journald option, as described in the Docker Logging Overview.

To collect events and statistics for the Sumo Logic App for Docker, first create Access Keys, then download and run the Sumo Logic Container, which includes the collector and the script source.

Create Access Keys 

Create Access Keys within Sumo Logic to register your Collector using the instructions in Access Keys.

Download and Run the Sumo Logic Container

The Sumo Logic Container, which includes a Collector and a Script Source, is located in the public Sumo Logic Docker Hub. Download this container using the following procedure, then enter the access keys you created to register the collector.

To download and run the Container

  1. On your Docker host, download the Sumo Logic Container using the following command:

docker pull sumologic/collector:latest

  1. Run the container with this command, substituting the AccessID and AccessKey that you created previously in Sumo Logic:

docker run -d -v /var/run/docker.sock:/var/run/docker.sock --name="sumologic-docker" sumologic/collector:latest <AccessID> <AccessKey>

The container creates a collector in your Sumo Logic account named collector_container or collector_container-<ARandomString>.

It also creates two sources, a Docker Logs Source and a Docker Stats Source.

Once installed and configured, you can view the collector and its sources in Sumo Logic on the Manage > Collection page.

Configure a Source

On the Manage Data > Collection page, select Add > Add Source.

  1. Select Docker Logs or Docker Stats.
  2. Configure the Source fields:
    1. Name. (Required) A name is required. Description is optional. 
    2. URI. Enter the URI of the Docker daemon.
      • Same host (typically applies for Linux hosts). If your Collector agent runs on the same host as Docker containers, use the non-networked unix socket: unix:///var/run/docker.sock.
      • Remote access (typically applies for hosts on Mac or Windows where the docker process runs within a VM and the Collector agent runs outside of the Docker host). Run the docker-machine command to find the Docker environment variables.
      $ docker-machine env <machine-name>

      Example:

      $ docker-machine env default
      export DOCKER_TLS_VERIFY="1"
      export DOCKER_HOST="tcp://192.168.99.100:2376"
      export DOCKER_CERT_PATH="/Users/sumo/.docker/machine/machines/default"
      export DOCKER_MACHINE_NAME="default"
      # Run this command to configure your shell: 
      # eval "$(docker-machine env default)"

      ​​Change tcp to https in the DOCKER_HOST environment variable, e.g https://192.168.99.100:2376 is the URI.

    3. Cert Path. (Required for remote access only) Enter the path to the certificate files on the local machine where the Collector is running. Following the example above, the cert path is /Users/sumo/.docker/machine/machines/default
    4. Collect from/Containers. You can select all containers or enter a comma-separated list of specific containers. You can select all containers or enter a comma-separated list of specific containers. By default, you can collect from up to 40 containers. If you need to increase the limit, edit the collector.properties file and change the value of  docker.maxPerContainerConnections. The maximum supported value is 100.
    5. Source Host. Enter the hostname or IP address of the Source host. If not specified, it’s assumed that the host is the machine where Docker is running.  The hostname can be a maximum of 128 characters.
    6. Source Category. (Required) Enter the Sumo Logic Source category (such as prod/web/docker/stats). The Source Category metadata field is a fundamental building block to organize and label Sources. For details see Best Practices.
      Adding a Docker Source
  3. Configure the Advanced section:
    1. Enable Timestamp Parsing. True by default.
    2. Time Zone. By default, Use time zone from collector.
    3. Timestamp Format. Automatically detect the format by default.
    4. Encoding. UTF-8 by default.
    5. Enable Multiline Processing.
      • Detect Messages Spanning Multiple Lines. False
      • Infer Boundaries - Detect message boundaries automatically. False
      • Boundary Regex. None
  4. Click Save.

Sample Log Messages

{"status":"start", "id":"10adec58fa15202e06afef7b1b0b3b1464962a115ff56918444c3f22867d3f3b", "from":"hello-world", "time":1485975967}
{"status":"create", "id":"045599bc4d589264658f5f7f4efa3f1e3af9088ba1f7383a160cf344e1055d46", "from":"ubuntu", "time":1485966852}
{"read" : "2017-02-01T19:36:48.777487188Z", "network" : {"rx_bytes":87977,"rx_dropped":0,"rx_errors":0,"rx_packets":252,"tx_bytes":146194,"tx_dropped":0,"tx_errors":0,"tx_packets":302}, "cpu_stats" : {"cpu_usage":{"percpu_usage":[9469809313],"total_usage":9469809313,"usage_in_kernelmode":1050000000,"usage_in_usermode":8410000000},"system_cpu_usage":2496992710000000,"throttling_data":{"periods":0,"throttled_periods":0,"throttled_time":0}}, "blkio_stats" : {"io_merged_recursive":[],"io_queue_recursive":[],"io_service_bytes_recursive":[],"io_service_time_recursive":[],"io_serviced_recursive":[],"io_time_recursive":[],"io_wait_time_recursive":[],"sectors_recursive":[]}, "memory_stats" : {"limit":1033252864,"max_usage":202858496,"stats":{"active_anon":86831104,"active_file":13131776,"cache":24981504,"dirty":36864,"hierarchical_memory_limit":9223372036854771712,"inactive_anon":86786048,"inactive_file":11849728,"mapped_file":6430720,"pgfault":63351,"pgmajfault":146,"pgpgin":68526,"pgpgout":20040,"rss":173617152,"rss_huge":0,"total_active_anon":86831104,"total_active_file":13131776,"total_cache":24981504,"total_dirty":36864,"total_inactive_anon":86786048,"total_inactive_file":11849728,"total_mapped_file":6430720,"total_pgfault":63351,"total_pgmajfault":146,"total_pgpgin":68526,"total_pgpgout":20040,"total_rss":173617152,"total_rss_huge":0,"total_unevictable":0,"total_writeback":0,"unevictable":0,"writeback":0},"usage":201818112}}

Query Sample

Containers Created or Started

_sourceCategory=docker  ("\"status\":\"create\"" or "\"status\":\"start\"")  id from
| parse "\"status\":\"*\"" as status, "\"id\":\"*\"" as container_id, "\"from\":\"*\"" as image
| count_distinct(container_id)

Sumo Logic App

Now that you have set up collection for Docker, install the Sumo Logic App for Docker to use the preconfigured searches and dashboards that provide insight into website visitor behavior patterns, monitors server operations, and assists in troubleshooting issues that span entire web server farms.