Logging in Ranger

Available logs

While working, each Ranger component outputs logs into several files. The default log path is /var/log/ranger/<component>/<file>, where <component> is a Ranger component name in lowercase and <file> is a log file name. It is only possible to view these logs on each host as they are not displayed in the Ranger Admin UI. For example, here’s how to see one of the Ranger Admin log files:

$ cat /var/log/ranger/admin/ranger_admin_sql.log

You can see the mapping of the Ranger components with the log files in the table below.

Component Log file Purpose

Admin

catalina.out

The Tomcat service run logs. It contains everything that is written to Tomcat’s system.out and system.err streams

access-localhost<date>

Tomcat access log for <date>

ranger_admin_sql.log

Log events of the DBService (see the schema in Ranger architecture) retrieval

gc-worker.log

The Ranger Admin garbage collection logs

ranger-admin-<host_fqdn>-ranger.log

The Ranger Admin component run logs

ranger_admin_perf.log

Ranger performance logs

ranger_db_patch.log

Patch log for upgrades, installation, and migrations

KMS

catalina.out

The Tomcat service run logs

kms-audit-<host_fqdn>-kms.log

The Ranger KMS component audit logs

ranger-kms-<host_fqdn>-kms.log

The Ranger KMS component run logs

UserSync

auth.log

Original service logs before the standard logger interception

usersync-<host_fqdn>-ranger.log

The Ranger UserSync component run logs

Log settings

Ranger uses the logback framework for logging. It’s a successor to widely known Log4j with better performance, native slf4j support, and extended filtration options.

Logback file structure

For example, the default logback.xml configuration file for Ranger KMS (with the license information, comments, and some duplicating tags removed) is presented below:

<?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true">
  <appender name="kms-audit" class="ch.qos.logback.core.rolling.RollingFileAppender"> (1)
    <Append>true</Append> (2)
    <File>${kms.log.dir}/kms-audit-${hostname}-${user}.log</File> (3)
    <encoder> (4)
      <pattern>%d{ISO8601} %m%n</pattern>
    </encoder>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> (5)
      <fileNamePattern>${kms.log.dir}/kms-audit-${hostname}-${user}.log.%d{yyyy-MM-dd}</fileNamePattern> (6)
      <maxHistory>15</maxHistory> (7)
      <cleanHistoryOnStart>true</cleanHistoryOnStart> (8)
    </rollingPolicy>
  </appender>
  <appender name="kms" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <File>${kms.log.dir}/ranger-kms-${hostname}-${user}.log</File>
    <Append>true</Append>
    <encoder>
      <pattern>%d{ISO8601} %-5p [%t] %c{1} \(%F:%L\) - %m%n</pattern>
    </encoder>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <fileNamePattern>${kms.log.dir}/ranger-kms-${hostname}-${user}.log.%d{yyyy-MM-dd}</fileNamePattern>
      <maxHistory>15</maxHistory>
      <cleanHistoryOnStart>true</cleanHistoryOnStart>
    </rollingPolicy>
  </appender>
  <logger name="kms-audit" additivity="false" level="INFO"> (9)
    <appender-ref ref="kms-audit"/> (10)
  </logger>
  <root level="WARN"> (11)
    <appender-ref ref="kms"/>
  </root>
</configuration>

The logback.xml file can have many sections, information about which you can find in the logback documentation, but here are descriptions of the used sections:

1 appender. The <appender> tag represents an object that contains all the information necessary for logging: output file, log format, what to log and when to log it. Logback offers several types of appenders, but by default only RollingFileAppender is used.
2 Append. The <Append> tag contains a boolean value that defines whether to append the log file or truncate it each time.
3 File. The <File> tag contains the name of the output log file. If the file doesn’t exist, it is created.
4 encoder. The <encoder> object transforms an event into a byte array. The <pattern> parameter inside contains a layout defining what the output should look like.
5 rollingPolicy. This policy is responsible for rollover which involves file moving and renaming. There are many types of rolling policies, but the one used in Ranger is TimeBasedRollingPolicy which defines the rolling policy based on time (e.g. day, week, etc.).
6 fileNamePattern. This parameter’s value consists of a filename and %d section which may contain the date and time pattern. If the pattern is omitted, the default yyyy-MM-dd pattern is used. The rollover period is derived from this parameter’s value.
7 maxHistory. Defines how many log files can be kept at the same time. When the value gets exceeded, the older files will be deleted asynchronously.
8 cleanHistoryOnStart. Defines whether to delete an archive when the appender starts up.
9 logger. A logger definition. Here, you can specify the logging level for this logger.
10 appender-ref. An appender for this logger to use.
11 root. The root logger definition.

Logging levels

Logback offers the following logging levels:

  • OFF — logging is turned off.

  • ERROR — logs all the errors or events that may lead to an unexpected behavior.

  • WARN — logs all the unusual events that may signal the unintended use.

  • INFO — logs all the irregular or rare operations such as configuration loading.

  • DEBUG — logs all the major events. Mostly used for error localization as the name suggests.

  • TRACE — logs all the events. You can use it to localize an error if the DEBUG level isn’t sufficient.

Each subsequent level includes the logs of all the previous levels.

Configuration in ADCM

Each Ranger component has a configuration setting called logback.xml, where you can see the current log format, output file location, log level for certain services, and maximum history records. To edit this file in ADCM, follow the steps below:

  1. On the Clusters page, select your ADPS cluster.

  2. Go to the Services tab and select Ranger.

  3. Head to the Components tab. On this tab, select the component, for which you want to edit the log settings.

  4. Edit the logback.xml parameter to your needs, save the configuration, and restart Ranger.

Found a mistake? Seleсt text and press Ctrl+Enter to report it