The logcompare operator allows you to compare two sets of logs: baseline (historical) and target (current). To run a LogCompare operation, you can use the LogCompare button on the Messages tab to generate a properly formatted query.
The syntax of the LogCompare operator is described here. For examples of syntax, see LogCompare Syntax.
Compare vs. LogCompare
The compare and logcompare operators are very similar in syntax and functionality, but they handle different types of data:
- compare is used for aggregated numeric data (e.g., for analyzing results from a group by query or a query with aggregation operators such as count, sum, avg, etc. )
- logcompare is used for log signature counts (used right after the first pipe).
... | logcompare timeshift -24h
Compare the result of a query with the result of the same query for a time range shifted by 24 hours.
... | logcompare timeshift -1d
Compare the result of a query with the result of the same query for a time range shifted by 1 day. (Same as previous example.)
... | logcompare start_time 2016-01-06T12:00:00-08:00 end_time 2016-01-07T12:00:00-08:00
Compare the result of a query with the result of the same query for a time range specified by start_time and end_time. This must be a valid time range.
_sourceHost=cluster-1| logcompare timeshift -0s baseline(_sourceHost=cluster-2)
Compare logs on two different hosts (cluster-1 and cluster-2) for the same time period.
New Hidden Fields
These fields are generated by the logcompare operator, and can be used in the following logcompare operator query string.
These fields are described in the following table:
The number of log messages that belong to this cluster for this query.
The percent change of the signature, calculated as (targetPercentage - baselinePercentage) / baselinePercentage, where baselinePercentage is the number of logs matched to the signature divided by the total number of logs in the baseline, and similarly for targetPercentage. This is infinity for new signatures.
The value is calculated using a symmetric version of Kullback-Leibler divergence score.
Values are Boolean.1 if the cluster is new, otherwise 0.
Using Hidden Fields
When you use the logcompare operator, new hidden fields are created that you can use to focus your results. Here's how you can use these fields.
Show only signatures that are missing in the baseline query:
For example, you can run the query:
error | logcompare timeshift -1d | where (_isNew)
The query results are constrained to new clusters only using the _isNew field.
Show only signatures that are missing in the target query:
And in this example, the logcompare operator shows only clusters that no longer include any messages:
error | logcompare timeshift -1d | where _count ==0