Partitions provide three primary functions:
About creating partitions
To create a partition in an index, you will create a routing expression, which is a kind of query. A partition routing expression can take anything in a regular search query up to the first pipe—in other words, the search constraints. Partitions must be named alphanumerically, with no special characters, with the exception of underscores ( _ ). The query can include wild cards, but cannot include any parsing or search operators.
Create partitions for use cases that are not too general. The idea is to use partitions in an index to restrict your search for security and in order to improve search performance. If you create a partition for a very general use case, it would still work, you just wouldn’t benefit as much from increased performance.
Partitions ingest your messages in real time, and differ from scheduled views in that partitions don't backfill with aggregate data. Partitions begin building a non-aggregate index from the time the partition is created and only index data moving forward (from the time of creation).
Best practices for optimum performance
When designing partitions, keep the following in mind:
- Avoid using queries that are subject to change. In order to benefit from using partitions, they should be used for long-term message organization.
- Make the query as specific as possible. Making the query specific will reduce the amount of data in the partition, which increases search performance.
- Keep the query flexible. Use a flexible query, such as
sourceCategory=*Apache*, so that metadata can be adjusted without breaking the query.
- Group data together that is most often used together. For example, create partitions for categories such as web data, security data, or errors.
- Group data together that is used by teams. Partitions are an excellent way to organize messages by role and teams within your organization.
- Avoid including too much data in your Partition. Send between 2% and 20% of your data to a partition. Including 90% of the data in your index in a partition won’t improve search performance.
- Don’t create overlapping partitions. With multiple partitions, messages could be duplicated if you create routing expressions that overlap. For example, if you have the following partitions, messages for
_sourceCategory=prod/Apachewould be duplicated as they would be stored in both partitions.
Partitions and Data Tiers
If you have a Sumo Logic Enterprise Suite account, you can take advance of the Data Tiers feature, which allows you to locate the partition in the Frequent or Infrequent tier. You select the tier when you configure the partition. For more information, see Data Tiers.
Create a Partition
- In the Sumo left navigation bar, go to Manage Data > Logs, then select the Partitions tab.
- At the far right, click the plus sign (+). The Create Partition dialog appears.
- Partition Name. Enter a name for the partition.
- Routing Expression. Enter a keyword search expression that matches the data you want to have in the partition, using built-in metadata or custom metadata fields. If you have and Enterprise Suite account, and are going to assign. the partition to the Infrequent Tier, see the information in the "Assigning Data to a Data Tier" section of the Data Tiers page.
- Retention Period. Enter the number of days you wish to retain the data in the partition, or click Apply the retention period of the General Index.
- Data Tier. Click the radio button for the tier where you want the to partition to live.
- Data Forwarding. If you want to forward the data in the partition to a cloud environment, click Enable Data Forwarding and specify the necessary information for the options that appear. For more information Data Forwarding.