Log files collected using a Remote File Source must be encoded in UTF-8 or ASCII. If you are editing a Source, metadata changes are reflected going forward. Metadata for previously collected log data will not be retroactively changed.
Sumo Logic scans remote directories every 30 seconds.
To configure a Remote File Source:
- In Sumo Logic select Manage Data > Collection > Collection (Manage > Collection in the classic UI).
- Find the name of the installed Collector here you'd like to add a Source. Click Add and then choose Add Source from the pop-up menu.
- Select Remote File for the Source type.
- Set the following:
- Name. Type the name you'd like to display for the new Source. Description is optional. Source name metadata is stored in a searchable field called _sourceName.
- Host. Enter the hostname or the IP address of the remote machine (the hostname entered must be the system hostname or IP address and cannot be changed). The hostname is stored in a searchable field called _sourceHost. The hostname can be a maximum of 128 characters.
- Port. If your SSH server is listening on a nonstandard port, type the port number.
- Path Expression. Enter the absolute path expression to the file the Source should tail. Remote File Sources support wildcards in file paths. If the timestamp formats for the files are not identical, set up a separate Remote File Source for each file.
For Windows collections using Open SSH and Cygwin, specify the File path starting with/cygdrive. For example, if the path is "C:\mandy test\6.log" enter "/cygdrive/c/mandy\ test/6.log" in the File field. Use "\" to escape any spaces if they are present in the file path.
- Source Category. Type any string to tag the output collected from this Source with searchable metadata. For example, type firewall to tag all entries from this Source in a field called _sourceCategory. For more information, see Metadata Naming Conventions.
- Choose the type of Credentials used for this Source:
- Username and Password. Enter valid user credentials for the remote machine.
- Local SSH Config. Enter the username and the absolute path, including file name, to the PEM SSH key file located on the Collector host. Enter a password if required.
- Set any of the following under Advanced.
- Blacklist. Optional. Add any files to be excluded by including one or more path expressions separated by commas. Note that this field takes a maximum of 10240 characters.
- Enable Timestamp Parsing. This option is selected by default. If it's deselected, no timestamp information is parsed at all.
- Time Zone. There are two options for Time Zone. You can use the time zone present in your log files, and then choose an option in case time zone information is missing from a log message. Or, you can have Sumo Logic completely disregard any time zone information present in logs by forcing a time zone. It's very important to have the proper time zone set, no matter which option you choose. If the time zone of logs can't be determined, Sumo Logic assigns logs UTC; if the rest of your logs are from another time zone your search results will be affected.
- Timestamp Format. By default, Sumo Logic will automatically detect the timestamp format of your logs. However, you can manually specify a timestamp format for a Source. See Timestamps, Time Zones, Time Ranges, and Date Formats for more information.
- Enable Multiline Processing. Multiline processing is enabled by default. Use this option if you're working with multi-line messages (for example, log4J or exception stack traces). Deselect this option if you want to avoid unnecessary processing when collecting single-message-per-line files (for example, Linux system.log).
- Infer Boundaries. Enable when you want Sumo Logic to automatically attempt to determine which lines belong to the same message.
If you deselect the Infer Boundaries option, you will need to enter a regular expression in the Boundary Regex field to use for detecting the entire first line of multi-line messages.
- Boundary Regex. You can specify the boundary between messages using a regular expression. Enter a regular expression for the full first line of every multi-line message in your log files. For an example, see the information on boundary regex in Prerequisites for Remote Windows Event Log Collection.
- Create any processing rules you'd like for the new Source.
- When you are finished configuring the Source click Save.