Logging in Ranger
Available logs
While working, each Ranger component outputs logs into several files. The default log path is /var/log/ranger/<component>/<file>, where <component>
is a Ranger component name in lowercase and <file>
is a log file name. It is only possible to view these logs on each host as they are not displayed in the Ranger Admin UI. For example, here’s how to see one of the Ranger Admin log files:
$ cat /var/log/ranger/admin/ranger_admin_sql.log
You can see the mapping of the Ranger components with the log files in the table below.
Component | Log file | Purpose |
---|---|---|
Admin |
catalina.out |
The Tomcat service run logs. It contains everything that is written to Tomcat’s |
access-localhost<date> |
Tomcat access log for |
|
ranger_admin_sql.log |
Log events of the DBService (see the schema in Ranger architecture) retrieval |
|
gc-worker.log |
The Ranger Admin garbage collection logs |
|
ranger-admin-<host_fqdn>-ranger.log |
The Ranger Admin component run logs |
|
ranger_admin_perf.log |
Ranger performance logs |
|
ranger_db_patch.log |
Patch log for upgrades, installation, and migrations |
|
KMS |
catalina.out |
The Tomcat service run logs |
kms-audit-<host_fqdn>-kms.log |
The Ranger KMS component audit logs |
|
ranger-kms-<host_fqdn>-kms.log |
The Ranger KMS component run logs |
|
UserSync |
auth.log |
Original service logs before the standard logger interception |
usersync-<host_fqdn>-ranger.log |
The Ranger UserSync component run logs |
Log settings
Ranger uses the logback framework for logging. It’s a successor to widely known Log4j with better performance, native slf4j support, and extended filtration options.
Logback file structure
For example, the default logback.xml configuration file for Ranger KMS (with the license information, comments, and some duplicating tags removed) is presented below:
<?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true">
<appender name="kms-audit" class="ch.qos.logback.core.rolling.RollingFileAppender"> (1)
<Append>true</Append> (2)
<File>${kms.log.dir}/kms-audit-${hostname}-${user}.log</File> (3)
<encoder> (4)
<pattern>%d{ISO8601} %m%n</pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> (5)
<fileNamePattern>${kms.log.dir}/kms-audit-${hostname}-${user}.log.%d{yyyy-MM-dd}</fileNamePattern> (6)
<maxHistory>15</maxHistory> (7)
<cleanHistoryOnStart>true</cleanHistoryOnStart> (8)
</rollingPolicy>
</appender>
<appender name="kms" class="ch.qos.logback.core.rolling.RollingFileAppender">
<File>${kms.log.dir}/ranger-kms-${hostname}-${user}.log</File>
<Append>true</Append>
<encoder>
<pattern>%d{ISO8601} %-5p [%t] %c{1} \(%F:%L\) - %m%n</pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${kms.log.dir}/ranger-kms-${hostname}-${user}.log.%d{yyyy-MM-dd}</fileNamePattern>
<maxHistory>15</maxHistory>
<cleanHistoryOnStart>true</cleanHistoryOnStart>
</rollingPolicy>
</appender>
<logger name="kms-audit" additivity="false" level="INFO"> (9)
<appender-ref ref="kms-audit"/> (10)
</logger>
<root level="WARN"> (11)
<appender-ref ref="kms"/>
</root>
</configuration>
The logback.xml file can have many sections, information about which you can find in the logback documentation, but here are descriptions of the used sections:
1 | appender. The <appender> tag represents an object that contains all the information necessary for logging: output file, log format, what to log and when to log it. Logback offers several types of appenders, but by default only RollingFileAppender is used. |
2 | Append. The <Append> tag contains a boolean value that defines whether to append the log file or truncate it each time. |
3 | File. The <File> tag contains the name of the output log file. If the file doesn’t exist, it is created. |
4 | encoder. The <encoder> object transforms an event into a byte array. The <pattern> parameter inside contains a layout defining what the output should look like. |
5 | rollingPolicy. This policy is responsible for rollover which involves file moving and renaming. There are many types of rolling policies, but the one used in Ranger is TimeBasedRollingPolicy which defines the rolling policy based on time (e.g. day, week, etc.). |
6 | fileNamePattern. This parameter’s value consists of a filename and %d section which may contain the date and time pattern. If the pattern is omitted, the default yyyy-MM-dd pattern is used. The rollover period is derived from this parameter’s value. |
7 | maxHistory. Defines how many log files can be kept at the same time. When the value gets exceeded, the older files will be deleted asynchronously. |
8 | cleanHistoryOnStart. Defines whether to delete an archive when the appender starts up. |
9 | logger. A logger definition. Here, you can specify the logging level for this logger. |
10 | appender-ref. An appender for this logger to use. |
11 | root. The root logger definition. |
Logging levels
Logback offers the following logging levels:
-
OFF
— logging is turned off. -
ERROR
— logs all the errors or events that may lead to an unexpected behavior. -
WARN
— logs all the unusual events that may signal the unintended use. -
INFO
— logs all the irregular or rare operations such as configuration loading. -
DEBUG
— logs all the major events. Mostly used for error localization as the name suggests. -
TRACE
— logs all the events. You can use it to localize an error if theDEBUG
level isn’t sufficient.
Each subsequent level includes the logs of all the previous levels.
Configuration in ADCM
Each Ranger component has a configuration setting called logback.xml, where you can see the current log format, output file location, log level for certain services, and maximum history records. To edit this file in ADCM, follow the steps below:
-
On the Clusters page, select your ADPS cluster.
-
Go to the Services tab and select Ranger.
-
Head to the Components tab. On this tab, select the component, for which you want to edit the log settings.
-
Edit the logback.xml parameter to your needs, save the configuration, and restart Ranger.