Skip to main content

Performance Benchmarks

Logs collection

The following benchmark has been compiled on an Amazon m4.large instance, which has 2 CPU cores and 8 GB of memory available.

CPU benchmarking was done using filelogreceiver.

CPU usage guidelines

Benchmark - CPU usage for particular average message size and EPS

Measured CPU usage for particular Events Per Second (EPS) average message size.

100B512B1KB5KB10KB
EPS-----
1001.14%1%1.01%1.4%3.78%
2001.29%1.4%1.41%2.57%5.36%
5002.75%2.71%2.95%5.7%10.68%
10004.74%5.07%5.32%11.3%20.12%
15007.08%7.29%7.99%16.93%27.96%
20009.64%9.56%10.39%22.51%36.59%

Benchmark - EPS for average message size and CPU usage

Events Per Second (EPS) achieved for a particular average message size and CPU usage.

100B512B1KB5KB10KB
Average CPU usage-----
5%200011001000150200
10%350021001500450300
20%6500410030001200700
50%14000101008500--
90%-19100---

Using the information from the above table, if you had an average CPU usage of 5%:

  • 10 KB logs can be ingested at 200 logs/sec (2000 KB/sec).
  • 1 KB logs can be ingested at 1000 logs/sec (1000 KB/sec).

This shows that the collector performs better when it is made to ingest bigger log entries (which is expected due to less overhead coming from timestamp parsing, etc.).

Memory usage guidelines

Benchmark - memory usage for particular average message size and EPS

Measured memory usage (in MB) for particular Events Per Second (EPS) average message size.

100B512B1KB5KB10KB
EPS-----
100113.14116.16117.1116.99112.59
200115.16118.55116.8119.67127.02
500118.24121.79122.78127.87142.73
1000121.6126.75127.94140.11106.82
1500128.54131.9137.6995.21113.89
2000130.62125.27144.5998.62134.61

Fine Tuning Performance

There are a couple configuration options that can help with performance in specific scenarios.

Sumo Logic Exporter

The Sumo Logic Exporter sends data to Sumo Logic. It has the following features that can help with performance:

  • retry_on_failure with its initial_interval, max_interval and max_elapsed_time settings
  • sending_queue with its num_consumers, queue_size settings
  • timeout

Read more about these features in the Sumo Logic Exporter docs.

Batch Processor

The Batch Processor joins records of each type in batches. It has the following features that can help with performance:

  • send_batch_size
  • send_batch_max_size
  • timeout

Read more about these features in the Batch Processor docs.

Memory Limiter Processor

The Memory Limiter Processor prevents out-of-memory crashes for the collector process by monitoring the amount of memory used by the collector and forcing it to lower its memory consumption.

Read more about its features in the Memory Limiter Processor docs.

Status
Legal
Privacy Statement
Terms of Use

Copyright © 2024 by Sumo Logic, Inc.