Question:

The collector.log has a prominent Java exception that is preventing the Sumo Logic Collector from uploading Source log data to the Sumo Logic service.

2013-10-14 08:13:32,190 [Blade000000000046F010-Scanner] WARN com.sumologic.scala.collector.blade.wildcard.LocalDirectoryBladeScanner - Unexpected exception 
in scanner or downstream pipeline 
java.io.IOException: Too many levels of symbolic links

Why does this error occur? 

Answer:

There are typically two reason why this error could occur:

  1.  You have more than 32 nested symbolic links within a path expression.

Symlink A --> Symlink B --> Symlink C --> Symlink D --> Etc..
  1. You have one symbolic link that is referencing back up to a parent symbolic link resulting in a circular reference. In this case, after 32 tries the Collector throws an error.
Symlink A --> Symlink B
Symlink B --> Symlink C
Symlink C --> Symlink A

 

This error can be corrected by either reducing the number of symbolic link levels or by identifying and removing the symbolic link that references back to the parent. To identify the Source that is generating the error, use the following steps. 

  1. Take the Source ID from the log message, highlighted in this example.
     

2013-10-14 08:13:32,190 [Blade000000000046F010-Scanner] WARN
  1. Using the source ID, find and open the corresponding Source configuration file within the local Collector. The source configuration files are located within/<sumo_home>/config/blades and will be in the form of <sourceID>.json.
     
cat /<sumo_home>/config/blades/000000000046F010.json 
  1. The Source configuration includes the name of the Source and the path expression used. You can use this information to look for nested or circular symbolic links.
     
"blade.name":"My Source", "pathExpression":"/var/logs/linkA"