Конфигурационные параметры MiNiFi
Для конфигурации сервиса используйте следующие параметры, доступные в ADCM.
|
ПРИМЕЧАНИЕ
|
| Параметр | Описание | Значение по умолчанию |
|---|---|---|
MiNiFi C2 Server port |
HTTP-порт для подключения к MiNiFi C2 Server |
10080 |
nifi.minifi.notifier.ingestors.pull.http.query |
Cтрока запроса для извлечения конфигураций |
minifi |
| Параметр | Описание | Значение по умолчанию |
|---|---|---|
MINIFI_HOME |
Каталог для установки MiNiFi |
/usr/lib/minifi |
MINIFI_PID_DIR |
Каталог для хранения ID процесса MiNiFi |
/var/run/minifi |
MINIFI_LOG_DIR |
Каталог для хранения логов |
/var/log/minifi |
| Параметр | Описание | Значение по умолчанию |
|---|---|---|
MiNiFi Agent Heap size |
Heap-размер агента |
256m |
nifi.minifi.notifier.ingestors.pull.http.period.ms |
Период проверки обновлений (в мс) |
300000 |
nifi.minifi.status.reporter.log.query |
Запрос состояния экземпляра MiNiFi:
|
instance:
|
nifi.minifi.status.reporter.log.level |
Уровень журнала, на котором регистрируется статус |
INFO |
nifi.minifi.status.reporter.log.period |
Задержка между запросами (в мс) |
60000 |
nifi.minifi.security.keystore |
Полный путь и имя хранилища keystore |
— |
nifi.minifi.security.keystoreType |
Тип хранилища keystore |
— |
nifi.minifi.security.keystorePasswd |
Пароль хранилища keystore |
— |
nifi.minifi.security.keyPasswd |
Ключевой пароль |
— |
nifi.minifi.security.truststore |
Полный путь и имя хранилища truststore |
— |
nifi.minifi.security.truststoreType |
Тип хранилища truststore |
— |
nifi.minifi.security.truststorePasswd |
Пароль хранилища truststore |
— |
nifi.minifi.security.ssl.протокол |
Протокол безопасности |
— |
nifi.minifi.notifier.ingestors.pull.http.keystore.location |
Полный путь и имя хранилища keystore |
— |
nifi.minifi.notifier.ingestors.pull.http.keystore.type |
Тип хранилища keystore |
— |
nifi.minifi.notifier.ingestors.pull.http.keystore.password |
Пароль хранилища keystore |
— |
nifi.minifi.notifier.ingestors.pull.http.truststore.location |
Полный путь и имя хранилища truststore |
— |
nifi.minifi.notifier.ingestors.pull.http.truststore.type |
Тип хранилища truststore |
— |
nifi.minifi.notifier.ingestors.pull.http.truststore.password |
Пароль хранилища truststore |
— |
Настройка уровней логирования и log rotate для MiNiFi
| Параметр | Описание | Значение по умолчанию |
|---|---|---|
app_file_max_history |
Максимальное количество файлов для приложений |
10 |
boot_file_max_history |
Максимальное количество файлов для Boot |
5 |
status_file_max_history |
Максимальное количество файлов для статусов |
5 |
root_level |
Уровень события |
INFO |
Настройка структуры файла конфигурации логирования для MiNiFi
| Журнал (логгер) | Имена пакетов по умолчанию | Уровень события по умолчанию |
|---|---|---|
app_loggers |
org.apache.nifi |
INFO |
org.apache.nifi.processors |
WARN |
|
org.apache.nifi.processors.standard.LogAttribute |
INFO |
|
org.apache.nifi.controller.repository.StandardProcessSession |
WARN |
|
bootstrap_loggers |
org.apache.nifi.bootstrap |
INFO |
org.apache.nifi.bootstrap.Command |
INFO |
|
org.apache.nifi.StdOut |
INFO |
|
org.apache.nifi.StdErr |
ERROR |
|
status_loggers |
org.apache.nifi.minifi.bootstrap.status.reporters.StatusLogger |
INFO |
Шаблон для пользовательской настройки файла MiNiFi Agent logback.xml.
Значение по умолчанию:
<?xml version="1.0" encoding="UTF-8"?>
<!--
Maintained by ADCM
-->
<configuration scan="true" scanPeriod="30 seconds">
<contextListener class="ch.qos.logback.classic.jul.LevelChangePropagator">
<resetJUL>true</resetJUL>
</contextListener>
{% set logback = services.minifi.config['minifi_agent_logback_content'] %}
<appender name="APP_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${org.apache.nifi.minifi.bootstrap.config.log.dir}/minifi-app.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<!--
For daily rollover, use 'app_%d.log'.
For hourly rollover, use 'app_%d{yyyy-MM-dd_HH}.log'.
To GZIP rolled files, replace '.log' with '.log.gz'.
To ZIP rolled files, replace '.log' with '.log.zip'.
-->
<fileNamePattern>${org.apache.nifi.minifi.bootstrap.config.log.dir}/minifi-app_%d{yyyy-MM-dd_HH}.%i.log.gz</fileNamePattern>
<!-- Keep 10 rolling periods worth of log files-->
<maxHistory>{{ logback.app_file_max_history }}</maxHistory>
<!-- Max size each log file will be-->
<maxFileSize>1MB</maxFileSize>
<!-- Provide a cap of 10 MB across all archive files -->
<totalSizeCap>10MB</totalSizeCap>
</rollingPolicy>
<immediateFlush>true</immediateFlush>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder>
</appender>
<appender name="BOOTSTRAP_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${org.apache.nifi.minifi.bootstrap.config.log.dir}/minifi-bootstrap.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!--
For daily rollover, use 'user_%d.log'.
For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'.
To GZIP rolled files, replace '.log' with '.log.gz'.
To ZIP rolled files, replace '.log' with '.log.zip'.
-->
<fileNamePattern>${org.apache.nifi.minifi.bootstrap.config.log.dir}/minifi-bootstrap_%d.log.gz</fileNamePattern>
<!-- Keep 5 rolling periods worth of logs-->
<maxHistory>{{ logback.boot_file_max_history }}</maxHistory>
</rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder>
</appender>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder>
</appender>
<appender name="STATUS_LOG_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${org.apache.nifi.minifi.bootstrap.config.log.dir}/minifi-status.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!--
For daily rollover, use 'user_%d.log'.
For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'.
To GZIP rolled files, replace '.log' with '.log.gz'.
To ZIP rolled files, replace '.log' with '.log.zip'.
-->
<fileNamePattern>${org.apache.nifi.minifi.bootstrap.config.log.dir}/minifi-status_%d.log</fileNamePattern>
<!-- keep 5 log files worth of history -->
<maxHistory>{{ logback.status_file_max_history }}</maxHistory>
</rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder>
</appender>
<!-- valid logging levels: TRACE, DEBUG, INFO, WARN, ERROR -->
{% for key, value in logback.app_loggers | dictsort -%}
<logger name="{{ key }}" level="{{ value }}"/>
{% endfor -%}
<!-- Logger for managing logging statements for jetty -->
<logger name="org.eclipse.jetty" level="INFO"/>
<!-- Suppress non-error messages due to excessive logging by class or library -->
<logger name="com.sun.jersey.spi.container.servlet.WebComponent" level="ERROR"/>
<logger name="com.sun.jersey.spi.spring" level="ERROR"/>
<logger name="org.springframework" level="ERROR"/>
<!-- Suppress non-error messages due to known warning about redundant path annotation (NIFI-574) -->
<logger name="com.sun.jersey.spi.inject.Errors" level="ERROR"/>
<!--
Logger for capturing Bootstrap logs and MiNiFi's standard error and standard out.
-->
{% for key, value in logback.bootstrap_loggers | dictsort -%}
<logger name="{{ key }}" level="{{ value }}" additivity="false">
<appender-ref ref="BOOTSTRAP_FILE"/>
{% if key == "org.apache.nifi.minifi.bootstrap.Command" %}
<appender-ref ref="CONSOLE" />
{% endif -%}
</logger>
{% endfor -%}
{% for key, value in logback.status_loggers | dictsort -%}
<logger name="{{ key }}" level="{{ value }}" additivity="false">
<appender-ref ref="STATUS_LOG_FILE" />
</logger>
{% endfor -%}
<root level="{{ logback.root_level }}">
<appender-ref ref="APP_FILE"/>
</root>
</configuration>
Шаблон для пользовательской настройки файла MiNiFi Agent state-management.xml.
Значение по умолчанию:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<!--
Maintained by ADCM
-->
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<!--
This file provides a mechanism for defining and configuring the State Providers
that should be used for storing state locally and across a NiFi cluster. In order
to use a specific provider, it must be configured here and its identifier
must be specified in the nifi.properties file.
-->
<stateManagement>
<!--
State Provider that stores state locally in a configurable directory. This Provider requires the following properties:
Directory - the directory to store components' state in. If the directory being used is a sub-directory of the NiFi installation, it
is important that the directory be copied over to the new version when upgrading NiFi.
-->
<local-provider>
<id>local-provider</id>
<class>org.apache.nifi.controller.state.providers.local.WriteAheadLocalStateProvider</class>
<property name="Directory">./state/local</property>
</local-provider>
<!--
State Provider that is used to store state in ZooKeeper. This Provider requires the following properties:
Root Node - the root node in ZooKeeper where state should be stored. The default is '/nifi', but it is advisable to change this to a different value if not using
the embedded ZooKeeper server and if multiple NiFi instances may all be using the same ZooKeeper Server.
Connect String - A comma-separated list of host:port pairs to connect to ZooKeeper. For example, myhost.mydomain:2181,host2.mydomain:5555,host3:6666
Session Timeout - Specifies how long this instance of NiFi is allowed to be disconnected from ZooKeeper before creating a new ZooKeeper Session. Default value is "30 seconds"
Access Control - Specifies which Access Controls will be applied to the ZooKeeper ZNodes that are created by this State Provider. This value must be set to one of:
- Open : ZNodes will be open to any ZooKeeper client.
- CreatorOnly : ZNodes will be accessible only by the creator. The creator will have full access to create children, read, write, delete, and administer the ZNodes.
This option is available only if access to ZooKeeper is secured via Kerberos or if a Username and Password are set.
Username - An optional username that can be used to assign Access Controls to ZNodes. ZooKeeper allows users to assign arbitrary usernames and passwords to ZNodes. These usernames
and passwords are not explicitly defined elsewhere but are simply associated with ZNodes, so it is important that all NiFi nodes in a cluster have the same value for the
Username and Password properties.
Password - An optional password that can be used to assign Access Controls to ZNodes. This property must be set if the Username property is set. NOTE: ZooKeeper transmits passwords
in plain text. As a result, a Username and Password should be used only if communicate with a ZooKeeper on a localhost or over encrypted comms (such as configuring SSL
communications with ZooKeeper).
-->
<cluster-provider>
<id>zk-provider</id>
<class>org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider</class>
<property name="Connect String"></property>
<property name="Root Node">/nifi</property>
<property name="Session Timeout">30 seconds</property>
<property name="Access Control">CreatorOnly</property>
<property name="Username">nifi</property>
<property name="Password">nifi</property>
</cluster-provider>
</stateManagement>
| Параметр | Описание | Значение по умолчанию |
|---|---|---|
minifi.c2.server.secure |
MiNiFi C2 безопасен |
— |
minifi.c2.server.keystore |
Полный путь и имя хранилища ключей keystore |
— |
minifi.c2.server.keystoreType |
Тип хранилища ключей keystore |
— |
minifi.c2.server.keystorePasswd |
Пароль хранилища ключей keystore |
— |
minifi.c2.server.keyPasswd |
Ключевой пароль |
— |
minifi.c2.server.truststore |
Полный путь и имя хранилища truststore |
— |
minifi.c2.server.truststoreType |
Тип хранилища truststore |
— |
minifi.c2.server.truststorePasswd |
Пароль хранилища truststore |
— |
Add key, value |
Введенные в этом поле параметры и их значения перезаписывают параметры, указанные в пользовательском интерфейсе ADCM. В этом поле также есть возможность определить значения для всех пользовательских параметров, которые не отображаются в интерфейсе, но разрешены в файле конфигурации MiNiFi C2 Server c2.properties |
— |
Настройка уровней логирования и ротации логов для MiNiFi C2 Server
| Параметр | Описание | Значение по умолчанию |
|---|---|---|
log_file_max_history |
Максимальное количество файлов для приложений |
5 |
root_level |
Уровень события |
INFO |
Настройка структуры файла конфигурации логирования для MiNiFi C2 Server
| Журнал (логгер) | Имена пакетов по умолчанию | Уровень события по умолчанию |
|---|---|---|
log_file_loggers |
org.apache.nifi.minifi.c2 |
DEBUG |
Шаблон для пользовательской настройки файла MiNiFi C2 Server authorizations.yaml.
Значение по умолчанию:
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the \"License\"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an \"AS IS\" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
Default Action: deny
Paths:
/c2/config:
Default Action: deny
Actions:
- Authorization: CLASS_RASPI_3
Query Parameters:
class: raspi3
Action: allow
- Authorization: ROLE_SUPERUSER
Action: allow
# Default authorization lets anonymous pull any config. Remove below to change that.
- Authorization: ROLE_ANONYMOUS
Action: allow
/c2/config/contentTypes:
Default Action: deny
Actions:
- Authorization: CLASS_RASPI_3
Action: allow
# Default authorization lets anonymous pull any config. Remove below to change that.
- Authorization: ROLE_ANONYMOUS
Action: allow
/c2/config/heartbeat:
Default Action: deny
Actions:
- Authorization: CLASS_RASPI_3
Query Parameters:
class: raspi3
Action: allow
- Authorization: ROLE_SUPERUSER
Action: allow
# Default authorization lets anonymous pull any config. Remove below to change that.
- Authorization: ROLE_ANONYMOUS
Action: allow
/c2/config/acknowledge:
Default Action: deny
Actions:
- Authorization: CLASS_RASPI_3
Query Parameters:
class: raspi3
Action: allow
- Authorization: ROLE_SUPERUSER
Action: allow
# Default authorization lets anonymous pull any config. Remove below to change that.
- Authorization: ROLE_ANONYMOUS
Action: allow
Шаблон для пользовательской настройки файла MiNiFi C2 Server logback.xml.
Значение по умолчанию:
<?xml version="1.0" encoding="UTF-8"?>
<!--
Maintained by ADCM
-->
{% set logback = services.minifi.config['minifi_c2_server_logback_content'] -%}
<configuration scan="true" scanPeriod="30 seconds">
<contextListener class="ch.qos.logback.classic.jul.LevelChangePropagator">
<resetJUL>true</resetJUL>
</contextListener>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder>
</appender>
<appender name="LOG_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>/var/log/minifi-c2/minifi-c2.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>/var/log/minifi-c2/minifi-c2_%d.log</fileNamePattern>
<!-- keep 5 log files worth of history -->
<maxHistory>{{ logback.log_file_max_history }}</maxHistory>
</rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder>
</appender>
{% for key, value in logback.log_file_loggers | dictsort -%}
<logger name="{{ key }}" level="{{ value }}">
<appender-ref ref="LOG_FILE"/>
{% if key == "org.apache.nifi.minifi.c2" %}
<appender-ref ref="CONSOLE" />
{% endif -%}
</logger>
{% endfor -%}
<root level="{{ logback.root_level }}">
<appender-ref ref="LOG_FILE"/>
</root>
</configuration>
| Параметр | Описание | Значение по умолчанию |
|---|---|---|
Set service checks |
Проверка доступности после установки кластера |
true |