Конфигурационные параметры MiNiFi

Для конфигурации сервиса используйте следующие параметры, доступные в ADCM.

ПРИМЕЧАНИЕ
  • Некоторые из перечисленных параметров становятся доступны в UI ADCM после установки флага Advanced.

  • Значения, установленные в Custom-параметрах, перепишут текущие значения параметров, даже если они read-only.

Main

 

Параметр Описание Значение по умолчанию

MiNiFi C2 Server port

HTTP-порт для подключения к MiNiFi C2 Server

10080

nifi.minifi.notifier.ingestors.pull.http.query

Cтрока запроса для извлечения конфигураций

minifi

minifi-env.sh

 

Параметр Описание Значение по умолчанию

MINIFI_HOME

Каталог для установки MiNiFi

/usr/lib/minifi

MINIFI_PID_DIR

Каталог для хранения ID процесса MiNiFi

/var/run/minifi

MINIFI_LOG_DIR

Каталог для хранения логов

/var/log/minifi

MiNiFi Agent bootstrap.conf

 

Параметр Описание Значение по умолчанию

MiNiFi Agent Heap size

Heap-размер агента

256m

nifi.minifi.notifier.ingestors.pull.http.period.ms

Период проверки обновлений (в мс)

300000

nifi.minifi.status.reporter.log.query

Запрос состояния экземпляра MiNiFi:

  • health — состояние отчета экземпляра, активные потоки, наличие или отсутствие bulletins и каких-либо ошибок проверки;

  • bulletins — список всех текущих bulletins (если есть);

  • stats — текущее состояние экземпляра, включает в том числе байты чтения/записи и отправленные/переданные FlowFiles.

instance:

  • health;

  • bulletins.

nifi.minifi.status.reporter.log.level

Уровень журнала, на котором регистрируется статус

INFO

nifi.minifi.status.reporter.log.period

Задержка между запросами (в мс)

60000

nifi.minifi.security.keystore

Полный путь и имя хранилища keystore

 — 

nifi.minifi.security.keystoreType

Тип хранилища keystore

 — 

nifi.minifi.security.keystorePasswd

Пароль хранилища keystore

 — 

nifi.minifi.security.keyPasswd

Ключевой пароль

 — 

nifi.minifi.security.truststore

Полный путь и имя хранилища truststore

 — 

nifi.minifi.security.truststoreType

Тип хранилища truststore

 — 

nifi.minifi.security.truststorePasswd

Пароль хранилища truststore

 — 

nifi.minifi.security.ssl.протокол

Протокол безопасности

 — 

nifi.minifi.notifier.ingestors.pull.http.keystore.location

Полный путь и имя хранилища keystore

 — 

nifi.minifi.notifier.ingestors.pull.http.keystore.type

Тип хранилища keystore

 — 

nifi.minifi.notifier.ingestors.pull.http.keystore.password

Пароль хранилища keystore

 — 

nifi.minifi.notifier.ingestors.pull.http.truststore.location

Полный путь и имя хранилища truststore

 — 

nifi.minifi.notifier.ingestors.pull.http.truststore.type

Тип хранилища truststore

 — 

nifi.minifi.notifier.ingestors.pull.http.truststore.password

Пароль хранилища truststore

 — 

MiNiFi Agent logback.xml

 
Настройка уровней логирования и log rotate для MiNiFi

Параметр Описание Значение по умолчанию

app_file_max_history

Максимальное количество файлов для приложений

10

boot_file_max_history

Максимальное количество файлов для Boot

5

status_file_max_history

Максимальное количество файлов для статусов

5

root_level

Уровень события

INFO

Настройка структуры файла конфигурации логирования для MiNiFi

Журнал (логгер) Имена пакетов по умолчанию Уровень события по умолчанию

app_loggers

org.apache.nifi

INFO

org.apache.nifi.processors

WARN

org.apache.nifi.processors.standard.LogAttribute

INFO

org.apache.nifi.controller.repository.StandardProcessSession

WARN

bootstrap_loggers

org.apache.nifi.bootstrap

INFO

org.apache.nifi.bootstrap.Command

INFO

org.apache.nifi.StdOut

INFO

org.apache.nifi.StdErr

ERROR

status_loggers

org.apache.nifi.minifi.bootstrap.status.reporters.StatusLogger

INFO

MiNiFi Agent logback.xml template

 

    Шаблон для пользовательской настройки файла MiNiFi Agent logback.xml.

    Значение по умолчанию:

<?xml version="1.0" encoding="UTF-8"?>
<!--
  Maintained by ADCM
-->
<configuration scan="true" scanPeriod="30 seconds">
    <contextListener class="ch.qos.logback.classic.jul.LevelChangePropagator">
        <resetJUL>true</resetJUL>
    </contextListener>
    {% set logback = services.minifi.config['minifi_agent_logback_content'] %}
    <appender name="APP_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${org.apache.nifi.minifi.bootstrap.config.log.dir}/minifi-app.log</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
            <!--
              For daily rollover, use 'app_%d.log'.
              For hourly rollover, use 'app_%d{yyyy-MM-dd_HH}.log'.
              To GZIP rolled files, replace '.log' with '.log.gz'.
              To ZIP rolled files, replace '.log' with '.log.zip'.
            -->
            <fileNamePattern>${org.apache.nifi.minifi.bootstrap.config.log.dir}/minifi-app_%d{yyyy-MM-dd_HH}.%i.log.gz</fileNamePattern>
            <!-- Keep 10 rolling periods worth of log files-->
            <maxHistory>{{ logback.app_file_max_history }}</maxHistory>
            <!-- Max size each log file will be-->
            <maxFileSize>1MB</maxFileSize>
            <!-- Provide a cap of 10 MB across all archive files -->
            <totalSizeCap>10MB</totalSizeCap>
        </rollingPolicy>
        <immediateFlush>true</immediateFlush>
        <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
        </encoder>
    </appender>

    <appender name="BOOTSTRAP_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${org.apache.nifi.minifi.bootstrap.config.log.dir}/minifi-bootstrap.log</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <!--
              For daily rollover, use 'user_%d.log'.
              For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'.
              To GZIP rolled files, replace '.log' with '.log.gz'.
              To ZIP rolled files, replace '.log' with '.log.zip'.
            -->
            <fileNamePattern>${org.apache.nifi.minifi.bootstrap.config.log.dir}/minifi-bootstrap_%d.log.gz</fileNamePattern>
            <!-- Keep 5 rolling periods worth of logs-->
            <maxHistory>{{ logback.boot_file_max_history }}</maxHistory>
        </rollingPolicy>
        <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
        </encoder>
    </appender>

        <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
                <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
        </encoder>
        </appender>

    <appender name="STATUS_LOG_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${org.apache.nifi.minifi.bootstrap.config.log.dir}/minifi-status.log</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <!--
            For daily rollover, use 'user_%d.log'.
            For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'.
            To GZIP rolled files, replace '.log' with '.log.gz'.
            To ZIP rolled files, replace '.log' with '.log.zip'.
            -->
            <fileNamePattern>${org.apache.nifi.minifi.bootstrap.config.log.dir}/minifi-status_%d.log</fileNamePattern>
            <!-- keep 5 log files worth of history -->
            <maxHistory>{{ logback.status_file_max_history }}</maxHistory>
        </rollingPolicy>
        <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
        </encoder>
    </appender>


    <!-- valid logging levels: TRACE, DEBUG, INFO, WARN, ERROR -->

    {% for key, value in logback.app_loggers | dictsort -%}
    <logger name="{{ key }}" level="{{ value }}"/>
    {% endfor -%}

    <!-- Logger for managing logging statements for jetty -->
    <logger name="org.eclipse.jetty" level="INFO"/>

    <!-- Suppress non-error messages due to excessive logging by class or library -->
    <logger name="com.sun.jersey.spi.container.servlet.WebComponent" level="ERROR"/>
    <logger name="com.sun.jersey.spi.spring" level="ERROR"/>
    <logger name="org.springframework" level="ERROR"/>

    <!-- Suppress non-error messages due to known warning about redundant path annotation (NIFI-574) -->
    <logger name="com.sun.jersey.spi.inject.Errors" level="ERROR"/>

    <!--
        Logger for capturing Bootstrap logs and MiNiFi's standard error and standard out.
    -->

    {% for key, value in logback.bootstrap_loggers | dictsort -%}
    <logger name="{{ key }}" level="{{ value }}" additivity="false">
        <appender-ref ref="BOOTSTRAP_FILE"/>
    {% if key == "org.apache.nifi.minifi.bootstrap.Command" %}
        <appender-ref ref="CONSOLE" />
    {% endif -%}
    </logger>
    {% endfor -%}

    {% for key, value in logback.status_loggers | dictsort -%}
    <logger name="{{ key }}" level="{{ value }}" additivity="false">
        <appender-ref ref="STATUS_LOG_FILE" />
    </logger>
    {% endfor -%}

    <root level="{{ logback.root_level }}">
        <appender-ref ref="APP_FILE"/>
    </root>

</configuration>
MiNiFi Agent state-management.xml template

 

    Шаблон для пользовательской настройки файла MiNiFi Agent state-management.xml.

    Значение по умолчанию:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<!--
  Maintained by ADCM
-->
<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at
      http://www.apache.org/licenses/LICENSE-2.0
  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
-->
<!--
  This file provides a mechanism for defining and configuring the State Providers
  that should be used for storing state locally and across a NiFi cluster. In order
  to use a specific provider, it must be configured here and its identifier
  must be specified in the nifi.properties file.
-->
<stateManagement>
    <!--
        State Provider that stores state locally in a configurable directory. This Provider requires the following properties:

        Directory - the directory to store components' state in. If the directory being used is a sub-directory of the NiFi installation, it
                    is important that the directory be copied over to the new version when upgrading NiFi.
     -->
    <local-provider>
        <id>local-provider</id>
        <class>org.apache.nifi.controller.state.providers.local.WriteAheadLocalStateProvider</class>
        <property name="Directory">./state/local</property>
    </local-provider>

    <!--
        State Provider that is used to store state in ZooKeeper. This Provider requires the following properties:

        Root Node - the root node in ZooKeeper where state should be stored. The default is '/nifi', but it is advisable to change this to a different value if not using
                   the embedded ZooKeeper server and if multiple NiFi instances may all be using the same ZooKeeper Server.

        Connect String - A comma-separated list of host:port pairs to connect to ZooKeeper. For example, myhost.mydomain:2181,host2.mydomain:5555,host3:6666

        Session Timeout - Specifies how long this instance of NiFi is allowed to be disconnected from ZooKeeper before creating a new ZooKeeper Session. Default value is "30 seconds"

        Access Control - Specifies which Access Controls will be applied to the ZooKeeper ZNodes that are created by this State Provider. This value must be set to one of:
                            - Open  : ZNodes will be open to any ZooKeeper client.
                            - CreatorOnly  : ZNodes will be accessible only by the creator. The creator will have full access to create children, read, write, delete, and administer the ZNodes.
                                             This option is available only if access to ZooKeeper is secured via Kerberos or if a Username and Password are set.

        Username - An optional username that can be used to assign Access Controls to ZNodes. ZooKeeper allows users to assign arbitrary usernames and passwords to ZNodes. These usernames
                   and passwords are not explicitly defined elsewhere but are simply associated with ZNodes, so it is important that all NiFi nodes in a cluster have the same value for the
                   Username and Password properties.

        Password - An optional password that can be used to assign Access Controls to ZNodes. This property must be set if the Username property is set. NOTE: ZooKeeper transmits passwords
                   in plain text. As a result, a Username and Password should be used only if communicate with a ZooKeeper on a localhost or over encrypted comms (such as configuring SSL
                   communications with ZooKeeper).
    -->
    <cluster-provider>
        <id>zk-provider</id>
        <class>org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider</class>
        <property name="Connect String"></property>
        <property name="Root Node">/nifi</property>
        <property name="Session Timeout">30 seconds</property>
        <property name="Access Control">CreatorOnly</property>
        <property name="Username">nifi</property>
        <property name="Password">nifi</property>
    </cluster-provider>
</stateManagement>
MiNiFi C2 Server c2.properties

 

Параметр Описание Значение по умолчанию

minifi.c2.server.secure

MiNiFi C2 безопасен

 — 

minifi.c2.server.keystore

Полный путь и имя хранилища ключей keystore

 — 

minifi.c2.server.keystoreType

Тип хранилища ключей keystore

 — 

minifi.c2.server.keystorePasswd

Пароль хранилища ключей keystore

 — 

minifi.c2.server.keyPasswd

Ключевой пароль

 — 

minifi.c2.server.truststore

Полный путь и имя хранилища truststore

 — 

minifi.c2.server.truststoreType

Тип хранилища truststore

 — 

minifi.c2.server.truststorePasswd

Пароль хранилища truststore

 — 

Add key, value

Введенные в этом поле параметры и их значения перезаписывают параметры, указанные в пользовательском интерфейсе ADCM. В этом поле также есть возможность определить значения для всех пользовательских параметров, которые не отображаются в интерфейсе, но разрешены в файле конфигурации MiNiFi C2 Server c2.properties

 — 

MiNiFi C2 Server logback.xml

 

Настройка уровней логирования и ротации логов для MiNiFi C2 Server

Параметр Описание Значение по умолчанию

log_file_max_history

Максимальное количество файлов для приложений

5

root_level

Уровень события

INFO

Настройка структуры файла конфигурации логирования для MiNiFi C2 Server

Журнал (логгер) Имена пакетов по умолчанию Уровень события по умолчанию

log_file_loggers

org.apache.nifi.minifi.c2

DEBUG

MiNiFi C2 Server authorizations.yaml

 

    Шаблон для пользовательской настройки файла MiNiFi C2 Server authorizations.yaml.

    Значение по умолчанию:

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the \"License\"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an \"AS IS\" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

Default Action: deny
Paths:
  /c2/config:
    Default Action: deny
    Actions:
    - Authorization: CLASS_RASPI_3
      Query Parameters:
        class: raspi3
      Action: allow
    - Authorization: ROLE_SUPERUSER
      Action: allow

    # Default authorization lets anonymous pull any config.  Remove below to change that.
    - Authorization: ROLE_ANONYMOUS
      Action: allow

  /c2/config/contentTypes:
    Default Action: deny
    Actions:
    - Authorization: CLASS_RASPI_3
      Action: allow
    # Default authorization lets anonymous pull any config.  Remove below to change that.
    - Authorization: ROLE_ANONYMOUS
      Action: allow

  /c2/config/heartbeat:
    Default Action: deny
    Actions:
      - Authorization: CLASS_RASPI_3
        Query Parameters:
          class: raspi3
        Action: allow
      - Authorization: ROLE_SUPERUSER
        Action: allow

      # Default authorization lets anonymous pull any config.  Remove below to change that.
      - Authorization: ROLE_ANONYMOUS
        Action: allow

  /c2/config/acknowledge:
    Default Action: deny
    Actions:
      - Authorization: CLASS_RASPI_3
        Query Parameters:
          class: raspi3
        Action: allow
      - Authorization: ROLE_SUPERUSER
        Action: allow

      # Default authorization lets anonymous pull any config.  Remove below to change that.
      - Authorization: ROLE_ANONYMOUS
        Action: allow
MiNiFi C2 Server logback.xml template

 

    Шаблон для пользовательской настройки файла MiNiFi C2 Server logback.xml.

    Значение по умолчанию:

<?xml version="1.0" encoding="UTF-8"?>
<!--
  Maintained by ADCM
-->
{% set logback = services.minifi.config['minifi_c2_server_logback_content'] -%}
<configuration scan="true" scanPeriod="30 seconds">
    <contextListener class="ch.qos.logback.classic.jul.LevelChangePropagator">
        <resetJUL>true</resetJUL>
    </contextListener>
        <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
                <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
        </encoder>
        </appender>

    <appender name="LOG_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>/var/log/minifi-c2/minifi-c2.log</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>/var/log/minifi-c2/minifi-c2_%d.log</fileNamePattern>
            <!-- keep 5 log files worth of history -->
            <maxHistory>{{ logback.log_file_max_history }}</maxHistory>
        </rollingPolicy>
        <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
        </encoder>
    </appender>

    {% for key, value in logback.log_file_loggers | dictsort -%}
    <logger name="{{ key }}" level="{{ value }}">
        <appender-ref ref="LOG_FILE"/>
    {% if key == "org.apache.nifi.minifi.c2" %}
        <appender-ref ref="CONSOLE" />
    {% endif -%}
    </logger>
    {% endfor -%}

    <root level="{{ logback.root_level }}">
        <appender-ref ref="LOG_FILE"/>
    </root>

</configuration>
Параметр Описание Значение по умолчанию

Set service checks

Проверка доступности после установки кластера

true

Нашли ошибку? Выделите текст и нажмите Ctrl+Enter чтобы сообщить о ней