Skip to main content
Sumo Logic

Collect Logs for Artifactory

This procedure documents how to collect logs from JFrog Artifactory into Sumo Logic.

Log Types

Sumo Logic reads logs in the directory /var/opt/jfrog/artifactory/logs:

  • Artifactory Server (artifactory.log)
  • Access (access.log)
  • Request (request.log)
  • Traffic (traffic.*.log)

For more information about Artifactory logs, see JFrog's Artifactory Log Files.

Configure a Collector

Configure an Installed Collector.

Configure a Source

  1. Configure a Local File source.
  2. Configure the Source fields:
    1. Name. (Required) A name is required. Description is optional.
    2. Source Category. (Required) The Source Category metadata field is a fundamental building block to organize and label Sources. It must end with "artifactory/console" to work with the Artifactory app. Example: _sourceCategory=<Foo>/artifactory/console. For category naming, see Best Practices.
  3. Configure the Advanced section:
    1. Enable Timestamp Parsing. True
    2. Time Zone. Make sure to set it to (UTC) Etc/UTC
    3. Timestamp Format. Auto Detect
    4. Encoding Type. UTF-8
    5. Multi-line Parsing. Detect Messages Spanning Multiple Lines, Infer Boundaries
  4. Click Save.

When you install the collector and configure the source, configure the following source categories to collect the logs and populate the dashboards.

The following suffixes are required. For example, you could use _sourceCategory=<Foo>/artifactory/console, but the suffix artifactory/console must be used.

Log Location Source Category
Artifactory Server /var/opt/jfrog/artifactory/logs/artifactory.log _sourceCategory=artifactory/console
Access /var/opt/jfrog/artifactory/logs/access.log _sourceCategory=artifactory/access
Request /var/opt/jfrog/artifactory/logs/request.log _sourceCategory=artifactory/request
Traffic /var/opt/jfrog/artifactory/logs/traffic.*.log _sourceCategory=artifactory/traffic

Field Extraction Rules

Here are Artifactory extraction rules that use different approaches. 

Traffic

_sourceCategory=*artifactory*
| where _sourceCategory matches "*artifactory/traffic"
| parse regex "(?<year>\d{4})(?<month>\d{2})(?<day>\d{2})(?<hour>\d{2})(?<minute>\d{2})(?<second>\d{2})\|\d*\|(?<direction>[^|]*)\|\s*(?<ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|[^|]*)\|(?<repo>[^:]*):(?<fullfilepath>[^|]*)\|(?<size>\d*)" nodrop

Access Logs

_sourceCategory=*artifactory*
| where _sourceCategory matches "*artifactory/access"
| parse "[*] *:* for */*" as what, repo, path, user, ip

Request Logs

_sourceCategory=*artifactory*
| where _sourceCategory matches "*artifactory/request"
| parse "*|*|*|*|*|*|*|*|*|*" as datetime, response_time, type, ip, user, method, path, protocol, status_code, size

Sample Log Messages

20170113185444|17|REQUEST|1.1.1.1|anonymous|GET|/cloudera-repos/org/slf4j/slf4j-log4j12/1.7.5/slf4j-log4j12-1.7.5.jar|HTTP/1.1|200|8869
20170113185444|0|DOWNLOAD|1.1.1.1|cloudera-repos:org/apache/spark/spark-catalyst_2.11/2.0.1/spark-catalyst_2.11-2.0.1.jar.sha1|40
2017-01-13 18:54:12,121 [ACCEPTED DEPLOY] pypi-remote-cache:.pypi/test.html for billythekid/1.1.1.1.

Query Samples

Data Transfer Over Time

_sourceCategory=*artifactory*
| where _sourceCategory matches "*artifactory/traffic"
| parse regex
"(?<year>\d{4})(?<month>\d{2})(?<day>\d{2})(?<hour>\d{2})(?<minute>\d{2})(?<second>\d{2})\|\d*\|(?<direction>[^|]*)\|\s*(?<ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|[^|]*)\|(?<repo>[^:]*):(?<fullfilepath>[^|]*)\|(?<size>\d*)" nodrop
| timeslice 1h
| sum(size) by _timeslice, direction 
| _sum / (1024 * 1024 * 1024) as sizeinGB | sort by _sum
| fields -_sum
| transpose row _timeslice column direction

Requests by Status Code (Every 10 Minutes)

_sourcecategory=*artifactory*
| where _sourceCategory matches "*artifactory/request"
| parse "*|*|*|*|*|*|*|*|*|*" as datetime, response_time, type, ip, user, method, path, protocol, status_code, size
| timeslice 10m
| count _timeslice, status_code | sort by _count
| transpose row _timeslice column status_code

Unique Paths Accepted Deploys

_sourceCategory=*artifactory* "ACCEPTED DEPLOY" "-cache"
| where _sourceCategory matches "*artifactory/access"
| parse "[*] *:* for */*" as what, repo, path, user, ip
| parse regex field=ip "(?<ip>.*)\."
| where what = "ACCEPTED DEPLOY" and repo matches "*-cache"
| timeslice 10m
| count_distinct(path) as paths by _timeslice
| outlier paths

Sumo Logic App

Now that you have set up collection for Artifactory, install the Sumo Logic App for Artifactory to use the preconfigured searches and Dashboards that provide insight into website visitor behavior patterns, monitors server operations, and assists in troubleshooting issues that span entire web server farms.