MiNiFi configuration parameters
To configure the service, use the following configuration parameters in ADCM.
|
NOTE
|
| Parameter | Description | Default value |
|---|---|---|
MiNiFi C2 Server port |
HTTP port for connecting to MiNiFi C2 Server |
10080 |
nifi.minifi.notifier.ingestors.pull.http.query |
Query string to pull configuration |
minifi |
| Parameter | Description | Default value |
|---|---|---|
MINIFI_HOME |
Directory for installing MiNiFi |
/usr/lib/minifi |
MINIFI_PID_DIR |
Directory to store the MiNiFi process ID |
/var/run/minifi |
MINIFI_LOG_DIR |
Directory to store the logs |
/var/log/minifi |
| Parameter | Description | Default value |
|---|---|---|
MiNiFi Agent Heap size |
Agent heap size |
256m |
nifi.minifi.notifier.ingestors.pull.http.period.ms |
Update check period (in ms) |
300000 |
nifi.minifi.status.reporter.log.query |
Query the status of a MiNiFi instance:
|
instance:
|
nifi.minifi.status.reporter.log.level |
Logging level at which the status is logged |
INFO |
nifi.minifi.status.reporter.log.period |
Delay between each request (in ms) |
60000 |
nifi.minifi.security.keystore |
Full path and name of the keystore |
— |
nifi.minifi.security.keystoreType |
Keystore type |
— |
nifi.minifi.security.keystorePasswd |
Keystore password |
— |
nifi.minifi.security.keyPasswd |
Key password |
— |
nifi.minifi.security.truststore |
Full path and name of the truststore |
— |
nifi.minifi.security.truststoreType |
Truststore type |
— |
nifi.minifi.security.truststorePasswd |
Truststore password |
— |
nifi.minifi.security.ssl.protocol |
Security protocol |
— |
nifi.minifi.notifier.ingestors.pull.http.keystore.location |
Full path and name of the keystore |
— |
nifi.minifi.notifier.ingestors.pull.http.keystore.type |
Keystore type |
— |
nifi.minifi.notifier.ingestors.pull.http.keystore.password |
Keystore password |
— |
nifi.minifi.notifier.ingestors.pull.http.truststore.location |
Full path and name of the truststore |
— |
nifi.minifi.notifier.ingestors.pull.http.truststore.type |
Truststore type |
— |
nifi.minifi.notifier.ingestors.pull.http.truststore.password |
Truststore password |
— |
Setting logging levels and log rotate for MiNiFi
| Parameter | Description | Default value |
|---|---|---|
app_file_max_history |
Maximum number of files for applications |
10 |
boot_file_max_history |
Maximum number of files for Boot |
5 |
status_file_max_history |
Maximum number of files for statuses |
5 |
root_level |
Event Level |
INFO |
Setting the structure of the logging configuration file for MiNiFi
| Logger | Default package names | Default event level |
|---|---|---|
app_loggers |
org.apache.nifi |
INFO |
org.apache.nifi.processors |
WARN |
|
org.apache.nifi.processors.standard.LogAttribute |
INFO |
|
org.apache.nifi.controller.repository.StandardProcessSession |
WARN |
|
bootstrap_loggers |
org.apache.nifi.bootstrap |
INFO |
org.apache.nifi.bootstrap.Command |
INFO |
|
org.apache.nifi.StdOut |
INFO |
|
org.apache.nifi.StdErr |
ERROR |
|
status_loggers |
org.apache.nifi.minifi.bootstrap.status.reporters.StatusLogger |
INFO |
Template for customizing the MiNiFi Agent logback.xml file.
Default value:
<?xml version="1.0" encoding="UTF-8"?>
<!--
Maintained by ADCM
-->
<configuration scan="true" scanPeriod="30 seconds">
<contextListener class="ch.qos.logback.classic.jul.LevelChangePropagator">
<resetJUL>true</resetJUL>
</contextListener>
{% set logback = services.minifi.config['minifi_agent_logback_content'] %}
<appender name="APP_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${org.apache.nifi.minifi.bootstrap.config.log.dir}/minifi-app.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<!--
For daily rollover, use 'app_%d.log'.
For hourly rollover, use 'app_%d{yyyy-MM-dd_HH}.log'.
To GZIP rolled files, replace '.log' with '.log.gz'.
To ZIP rolled files, replace '.log' with '.log.zip'.
-->
<fileNamePattern>${org.apache.nifi.minifi.bootstrap.config.log.dir}/minifi-app_%d{yyyy-MM-dd_HH}.%i.log.gz</fileNamePattern>
<!-- Keep 10 rolling periods worth of log files-->
<maxHistory>{{ logback.app_file_max_history }}</maxHistory>
<!-- Max size each log file will be-->
<maxFileSize>1MB</maxFileSize>
<!-- Provide a cap of 10 MB across all archive files -->
<totalSizeCap>10MB</totalSizeCap>
</rollingPolicy>
<immediateFlush>true</immediateFlush>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder>
</appender>
<appender name="BOOTSTRAP_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${org.apache.nifi.minifi.bootstrap.config.log.dir}/minifi-bootstrap.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!--
For daily rollover, use 'user_%d.log'.
For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'.
To GZIP rolled files, replace '.log' with '.log.gz'.
To ZIP rolled files, replace '.log' with '.log.zip'.
-->
<fileNamePattern>${org.apache.nifi.minifi.bootstrap.config.log.dir}/minifi-bootstrap_%d.log.gz</fileNamePattern>
<!-- Keep 5 rolling periods worth of logs-->
<maxHistory>{{ logback.boot_file_max_history }}</maxHistory>
</rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder>
</appender>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder>
</appender>
<appender name="STATUS_LOG_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${org.apache.nifi.minifi.bootstrap.config.log.dir}/minifi-status.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!--
For daily rollover, use 'user_%d.log'.
For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'.
To GZIP rolled files, replace '.log' with '.log.gz'.
To ZIP rolled files, replace '.log' with '.log.zip'.
-->
<fileNamePattern>${org.apache.nifi.minifi.bootstrap.config.log.dir}/minifi-status_%d.log</fileNamePattern>
<!-- keep 5 log files worth of history -->
<maxHistory>{{ logback.status_file_max_history }}</maxHistory>
</rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder>
</appender>
<!-- valid logging levels: TRACE, DEBUG, INFO, WARN, ERROR -->
{% for key, value in logback.app_loggers | dictsort -%}
<logger name="{{ key }}" level="{{ value }}"/>
{% endfor -%}
<!-- Logger for managing logging statements for jetty -->
<logger name="org.eclipse.jetty" level="INFO"/>
<!-- Suppress non-error messages due to excessive logging by class or library -->
<logger name="com.sun.jersey.spi.container.servlet.WebComponent" level="ERROR"/>
<logger name="com.sun.jersey.spi.spring" level="ERROR"/>
<logger name="org.springframework" level="ERROR"/>
<!-- Suppress non-error messages due to known warning about redundant path annotation (NIFI-574) -->
<logger name="com.sun.jersey.spi.inject.Errors" level="ERROR"/>
<!--
Logger for capturing Bootstrap logs and MiNiFi's standard error and standard out.
-->
{% for key, value in logback.bootstrap_loggers | dictsort -%}
<logger name="{{ key }}" level="{{ value }}" additivity="false">
<appender-ref ref="BOOTSTRAP_FILE"/>
{% if key == "org.apache.nifi.minifi.bootstrap.Command" %}
<appender-ref ref="CONSOLE" />
{% endif -%}
</logger>
{% endfor -%}
{% for key, value in logback.status_loggers | dictsort -%}
<logger name="{{ key }}" level="{{ value }}" additivity="false">
<appender-ref ref="STATUS_LOG_FILE" />
</logger>
{% endfor -%}
<root level="{{ logback.root_level }}">
<appender-ref ref="APP_FILE"/>
</root>
</configuration>
Template for customizing the MiNiFi Agent state-management.xml file.
Default value:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<!--
Maintained by ADCM
-->
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<!--
This file provides a mechanism for defining and configuring the State Providers
that should be used for storing state locally and across a NiFi cluster. In order
to use a specific provider, it must be configured here and its identifier
must be specified in the nifi.properties file.
-->
<stateManagement>
<!--
State Provider that stores state locally in a configurable directory. This Provider requires the following properties:
Directory - the directory to store components' state in. If the directory being used is a sub-directory of the NiFi installation, it
is important that the directory be copied over to the new version when upgrading NiFi.
-->
<local-provider>
<id>local-provider</id>
<class>org.apache.nifi.controller.state.providers.local.WriteAheadLocalStateProvider</class>
<property name="Directory">./state/local</property>
</local-provider>
<!--
State Provider that is used to store state in ZooKeeper. This Provider requires the following properties:
Root Node - the root node in ZooKeeper where state should be stored. The default is '/nifi', but it is advisable to change this to a different value if not using
the embedded ZooKeeper server and if multiple NiFi instances may all be using the same ZooKeeper Server.
Connect String - A comma-separated list of host:port pairs to connect to ZooKeeper. For example, myhost.mydomain:2181,host2.mydomain:5555,host3:6666
Session Timeout - Specifies how long this instance of NiFi is allowed to be disconnected from ZooKeeper before creating a new ZooKeeper Session. Default value is "30 seconds"
Access Control - Specifies which Access Controls will be applied to the ZooKeeper ZNodes that are created by this State Provider. This value must be set to one of:
- Open : ZNodes will be open to any ZooKeeper client.
- CreatorOnly : ZNodes will be accessible only by the creator. The creator will have full access to create children, read, write, delete, and administer the ZNodes.
This option is available only if access to ZooKeeper is secured via Kerberos or if a Username and Password are set.
Username - An optional username that can be used to assign Access Controls to ZNodes. ZooKeeper allows users to assign arbitrary usernames and passwords to ZNodes. These usernames
and passwords are not explicitly defined elsewhere but are simply associated with ZNodes, so it is important that all NiFi nodes in a cluster have the same value for the
Username and Password properties.
Password - An optional password that can be used to assign Access Controls to ZNodes. This property must be set if the Username property is set. NOTE: ZooKeeper transmits passwords
in plain text. As a result, a Username and Password should be used only if communicate with a ZooKeeper on a localhost or over encrypted comms (such as configuring SSL
communications with ZooKeeper).
-->
<cluster-provider>
<id>zk-provider</id>
<class>org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider</class>
<property name="Connect String"></property>
<property name="Root Node">/nifi</property>
<property name="Session Timeout">30 seconds</property>
<property name="Access Control">CreatorOnly</property>
<property name="Username">nifi</property>
<property name="Password">nifi</property>
</cluster-provider>
</stateManagement>
| Parameter | Description | Default value |
|---|---|---|
minifi.c2.server.secure |
Defines whether MiNiFi C2 is secure |
— |
minifi.c2.server.keystore |
Full path and name of the keystore |
— |
minifi.c2.server.keystoreType |
Keystore type |
— |
minifi.c2.server.keystorePasswd |
Keystore password |
— |
minifi.c2.server.keyPasswd |
Key password |
— |
minifi.c2.server.truststore |
Full path and name of the truststore |
— |
minifi.c2.server.truststoreType |
Truststore type |
— |
minifi.c2.server.truststorePasswd |
Truststore password |
— |
Add key, value |
Parameters and their values entered in this field override the parameters specified in the ADCM user interface. This field also allows you to set values for all user parameters that are not displayed in the interface, but are allowed in the configuration file MiNiFi C2 Server c2.properties |
— |
| Parameter | Description | Default value |
|---|---|---|
log_file_max_history |
Maximum number of files for applications |
10 |
root_level |
Event Level |
INFO |
Setting the structure of the logging configuration file for MiNiFi C2 Server
| Logger | Default package names | Default event level |
|---|---|---|
log_file_loggers |
org.apache.nifi.minifi.c2 |
DEBUG |
Template for customizing the MiNiFi C2 Server authorizations.yaml file.
Default value:
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the \"License\"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an \"AS IS\" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
Default Action: deny
Paths:
/c2/config:
Default Action: deny
Actions:
- Authorization: CLASS_RASPI_3
Query Parameters:
class: raspi3
Action: allow
- Authorization: ROLE_SUPERUSER
Action: allow
# Default authorization lets anonymous pull any config. Remove below to change that.
- Authorization: ROLE_ANONYMOUS
Action: allow
/c2/config/contentTypes:
Default Action: deny
Actions:
- Authorization: CLASS_RASPI_3
Action: allow
# Default authorization lets anonymous pull any config. Remove below to change that.
- Authorization: ROLE_ANONYMOUS
Action: allow
/c2/config/heartbeat:
Default Action: deny
Actions:
- Authorization: CLASS_RASPI_3
Query Parameters:
class: raspi3
Action: allow
- Authorization: ROLE_SUPERUSER
Action: allow
# Default authorization lets anonymous pull any config. Remove below to change that.
- Authorization: ROLE_ANONYMOUS
Action: allow
/c2/config/acknowledge:
Default Action: deny
Actions:
- Authorization: CLASS_RASPI_3
Query Parameters:
class: raspi3
Action: allow
- Authorization: ROLE_SUPERUSER
Action: allow
# Default authorization lets anonymous pull any config. Remove below to change that.
- Authorization: ROLE_ANONYMOUS
Action: allow
Template for customizing the MiNiFi C2 Server logback.xml file.
Default value:
<?xml version="1.0" encoding="UTF-8"?>
<!--
Maintained by ADCM
-->
{% set logback = services.minifi.config['minifi_c2_server_logback_content'] -%}
<configuration scan="true" scanPeriod="30 seconds">
<contextListener class="ch.qos.logback.classic.jul.LevelChangePropagator">
<resetJUL>true</resetJUL>
</contextListener>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder>
</appender>
<appender name="LOG_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>/var/log/minifi-c2/minifi-c2.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>/var/log/minifi-c2/minifi-c2_%d.log</fileNamePattern>
<!-- keep 5 log files worth of history -->
<maxHistory>{{ logback.log_file_max_history }}</maxHistory>
</rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder>
</appender>
{% for key, value in logback.log_file_loggers | dictsort -%}
<logger name="{{ key }}" level="{{ value }}">
<appender-ref ref="LOG_FILE"/>
{% if key == "org.apache.nifi.minifi.c2" %}
<appender-ref ref="CONSOLE" />
{% endif -%}
</logger>
{% endfor -%}
<root level="{{ logback.root_level }}">
<appender-ref ref="LOG_FILE"/>
</root>
</configuration>
| Parameter | Description | Default value |
|---|---|---|
Set service checks |
Specifies whether to check availability after cluster installation |
true |