NiFi configuration parameters
To configure the service, use the following configuration parameters in ADCM.
|
NOTE
|
| Parameter | Description | Default value |
|---|---|---|
Nifi config encryption password |
Password from which to derive the key for encrypting the sensitive properties. Must be at least 12 characters long |
0123456789ABC |
NiFi Ranger plugin enabled |
Indicates whether Ranger NiFi plugin is enabled (auto-populated) |
false |
Nifi-Registry Flow Provider |
Name of the active Flow Persistence Provider. Possible values:
|
FileSystemFlowPersistenceProvider |
Nifi-Registry Bundle Provider |
Name of the active Bundle Persistence Provider. Possible values:
|
FileSystemBundlePersistenceProvider |
Set service checks |
Defines whether to check availability after cluster installation |
true |
| Parameter | Description | Default value |
|---|---|---|
Nifi UI port |
NiFi Server HTTP port. Specified as property |
9090 |
Nifi server Heap size |
Heap size for NiFi Server. Specified in bootstrap.conf configuration file |
1024m |
Nifi Registry UI port |
NiFi Registry HTTP port. Specified as the |
18080 |
Nifi Registry Heap size |
Heap size for NiFi Registry. Specified in the bootstrap.conf configuration file |
512m |
nifi.queue.backpressure.count |
Default value for the number of FlowFile files (underlying NiFi processing object) that can be queued before backpressure is applied, i.e. the source stops sending data. The value must be an integer |
10000 |
nifi.queue.backpressure.size |
Default value for the maximum amount of data that must be queued before backpressure is applied. The value must be the size of the data, including the unit of measure |
1 GB |
java.arg.snappy |
Argument for the snappy library (the tempdir filesystem must be mounted with the |
-Dorg.xerial.snappy.tempdir=/var/tmp |
NiFi service repositories location options.
| Parameter | Description | Default value |
|---|---|---|
nifi.flowfile.repository.directory |
FlowFile repository location |
/usr/lib/nifi-server/flowfile_repository |
nifi.content.repository.directory |
Content repository location |
/usr/lib/nifi-server/content_repository |
nifi.provenance.repository.directory |
Provenance repository location |
/usr/lib/nifi-server/provenance_repository |
nifi.database.directory |
H2 database directory location |
/usr/lib/nifi-server/database_repository |
nifi.registry.db.directory |
Location of the Registry database directory |
/usr/lib/nifi-registry/database |
nifi.nar.library.directory.lib |
Parameter should be used in case of adding custom nars |
— |
| Parameter | Description | Default value |
|---|---|---|
xasecure.audit.destination.solr.batch.filespool.dir |
Directory for Solr audit spool |
/srv/ranger/nifi_plugin/audit_solr_spool |
xasecure.audit.destination.solr.urls |
Specifies Solr URL. Not setting when using ZooKeeper to connect to Solr |
— |
xasecure.audit.destination.solr.zookeepers |
ZooKeeper connection string for the Solr destination |
— |
xasecure.audit.destination.solr.force.use.inmemory.jaas.config |
ZooKeeper connections to Solr using configuration in a JAAS file |
— |
xasecure.audit.jaas.Client.loginModuleControlFlag |
Specifies whether the success of the module is required, requisite, sufficient, or optional |
— |
xasecure.audit.jaas.Client.loginModuleName |
Class name of the authentication technology used |
— |
xasecure.audit.jaas.Client.option.keyTab |
Set this to the file name of the keytab to get principal’s secret key |
— |
xasecure.audit.jaas.Client.option.serviceName |
Service name |
— |
xasecure.audit.jaas.Client.option.storeKey |
Enable if you want the keytab or the principal’s key to be stored in the Subject’s private credentials |
— |
xasecure.audit.jaas.Client.option.useKeyTab |
Enable if you want the module to get the principal’s key from the the keytab |
— |
Add key, value |
Parameters and their values entered in this field override the parameters specified in the ADCM user interface. This field also allows you to set values for all user parameters that are not displayed in the interface, but are allowed in the configuration file ranger-nifi-audit.xml |
— |
| Parameter | Description | Default value |
|---|---|---|
ranger.plugin.nifi.policy.rest.url |
URL to Ranger Admin |
— |
ranger.plugin.nifi.service.name |
Name of the Ranger service containing policies for this NiFi instance |
— |
ranger.plugin.nifi.policy.source.impl |
Class to retrieve policies from the source |
org.apache.ranger.admin.client.RangerAdminRESTClient |
ranger.plugin.nifi.policy.cache.dir |
Directory where Ranger policies are cached after successful retrieval from the source |
/srv/ranger/nifi/policycache |
ranger.plugin.nifi.policy.pollIntervalMs |
How often to poll for changes in policies |
30000 |
ranger.plugin.nifi.policy.rest.client.connection.timeoutMs |
NiFi plugin RangerRestClient connection timeout in milliseconds |
120000 |
ranger.plugin.nifi.policy.rest.client.read.timeoutMs |
NiFi plugin RangerRestClient read timeout in milliseconds |
30000 |
ranger.plugin.nifi.policy.rest.ssl.config.file |
Path to the file containing SSL details to contact Ranger Admin |
/etc/nifi/conf/ranger-nifi-policymgr-ssl.xml |
Add key, value |
Parameters and their values entered in this field override the parameters specified in the ADCM user interface. This field also allows you to set values for all user parameters that are not displayed in the interface, but are allowed in the configuration file ranger-nifi-security.xml |
— |
| Parameter | Description | Default value |
|---|---|---|
xasecure.policymgr.clientssl.keystore |
Location of the keystore file |
— |
xasecure.policymgr.clientssl.keystore.password |
Keystore password |
— |
xasecure.policymgr.clientssl.truststore |
Location of the truststore file |
— |
xasecure.policymgr.clientssl.truststore.password |
Truststore password |
— |
xasecure.policymgr.clientssl.keystore.credential.file |
Location of the keystore password credential file |
/etc/nifi/conf/keystore.jceks |
xasecure.policymgr.clientssl.truststore.credential.file |
Location of the truststore password credential file |
/etc/nifi/conf/truststore.jceks |
Add key, value |
Parameters and their values entered in this field override the parameters specified in the ADCM user interface. This field also allows you to set values for all user parameters that are not displayed in the interface, but are allowed in the configuration file ranger-nifi-policymgr-ssl.xml |
— |
| Parameter | Description | Default value |
|---|---|---|
xasecure.audit.destination.solr.batch.filespool.dir |
Directory for Solr audit spool |
/srv/ranger/nifi_registry_plugin/audit_solr_spool |
xasecure.audit.destination.solr.urls |
Specifies Solr URL |
— |
xasecure.audit.destination.solr.zookeepers |
Zookeeper connection string for the Solr destination |
— |
xasecure.audit.destination.solr.force.use.inmemory.jaas.config |
ZooKeeper connections to Solr using configuration in a JAAS file |
— |
xasecure.audit.jaas.Client.loginModuleControlFlag |
Specifies whether the success of the module is required, requisite, sufficient, or optional |
— |
xasecure.audit.jaas.Client.loginModuleName |
Class name of the authentication technology used |
— |
xasecure.audit.jaas.Client.option.keyTab |
Set this to the file name of the keytab to get principal’s secret key |
— |
xasecure.audit.jaas.Client.option.serviceName |
Service name |
— |
xasecure.audit.jaas.Client.option.storeKey |
Set this to |
— |
xasecure.audit.jaas.Client.option.useKeyTab |
Set this to |
— |
Add key, value |
Parameters and their values entered in this field override the parameters specified in the ADCM user interface. This field also allows you to set values for all user parameters that are not displayed in the interface, but are allowed in the configuration file ranger-nifi-registry-audit.xml |
— |
| Parameter | Description | Default value |
|---|---|---|
ranger.plugin.nifi-registry.policy.rest.url |
Path to the NiFi Registry variable for the Ranger service |
— |
ranger.plugin.nifi-registry.service.name |
Name of the Ranger service containing policies for this NiFi Registry instance |
— |
ranger.plugin.nifi-registry.policy.source.impl |
Class to retrieve policies from the source |
org.apache.ranger.admin.client.RangerAdminRESTClient |
ranger.plugin.nifi-registry.policy.cache.dir |
Directory where Ranger policies are cached after successful retrieval from the source |
/srv/ranger/nifi-registry/policycache |
ranger.plugin.nifi-registry.policy.pollIntervalMs |
How often to poll for changes in policies (in ms) |
30000 |
ranger.plugin.nifi-registry.policy.rest.client.connection.timeoutMs |
NiFi Registry plugin RangerRestClient connection timeout (in ms) |
120000 |
ranger.plugin.nifi-registry.policy.rest.client.read.timeoutMs |
NiFi Registrу plugin RangerRestClient read timeout (in ms) |
30000 |
ranger.plugin.nifi-registry.policy.rest.ssl.config.file |
Path to the file containing SSL details to contact Ranger Admin |
/etc/nifi-registry/conf/ranger-policymgr-ssl.xml |
Add key, value |
Parameters and their values entered in this field override the parameters specified in the ADCM user interface. This field also allows you to set values for all user parameters that are not displayed in the interface, but are allowed in the configuration file ranger-nifi-registry-security.xml |
— |
| Parameter | Description | Default value |
|---|---|---|
xasecure.policymgr.clientssl.keystore |
Location of the keystore file |
— |
xasecure.policymgr.clientssl.keystore.password |
Keystore password |
— |
xasecure.policymgr.clientssl.truststore |
Location of the truststore file |
— |
xasecure.policymgr.clientssl.truststore.password |
Truststore password |
— |
xasecure.policymgr.clientssl.keystore.credential.file |
Location of keystore password credential file |
/etc/nifi-registry/conf/keystore.jceks |
xasecure.policymgr.clientssl.truststore.credential.file |
Location of the truststore password credential file |
/etc/nifi-registry/conf/truststore.jceks |
Add key, value |
Parameters and their values entered in this field override the parameters specified in the ADCM user interface. This field also allows you to set values for all user parameters that are not displayed in the interface, but are allowed in the configuration file ranger-nifi-registry-policymgr-ssl.xml |
— |
| Parameter | Description | Default value |
|---|---|---|
DN NiFi’s nodes list |
List of user and system identifications to seed the User File. These are required fields to enable SSL for the first time.
Must include not only the DName of the NiFi Server component, but also the DName of the NiFi Registry, as well as the DName for the MiNiFi service components. For example, for an SSL-enabled cluster consisting of only NiFi Server, when adding a MiNiFi service or Schema Registry extension, you need to supplement this list with new DNames
. Example for nodes — |
— |
NiFi Initial Admin Identity |
The Initial Admin Identity which will be recorded in authorizers.xml and propagated to users.xml and authorizations.xml |
— |
NiFi Initial Admin Username |
The actual Initial Admin username which will be used to login to NiFi API during bundle checks |
— |
NiFi Initial Admin password |
Initial Admin password — password of the user designated by NiFi Initial Admin |
— |
Ranger Admin Identitity |
DN of the certificate that Ranger will use to communicate with NiFi. Requires a generated SSL keystore and truststore on the Ranger host. Affected only for NiFi Ranger Plugin |
— |
| Parameter | Description | Default value |
|---|---|---|
Authentication Strategy |
How the connection to the LDAP server is authenticated |
ANONYMOUS |
Manager DN |
DN of a user that has an entry in the Active Directory with right to search users and groups. Will be used to bind to an LDAP server to search for users |
— |
Manager Password |
Password of the manager that is used to bind to the LDAP server to search for users |
— |
TLS - Keystore |
Path to the keystore that is used when connecting to LDAP via LDAPS or START_TLS |
— |
TLS - Keystore Password |
Password for the keystore that is used when connecting to LDAP using LDAPS or START_TLS |
— |
TLS - Keystore Type |
Type of the keystore that is used when connecting to LDAP using LDAPS or START_TLS (i.e. JKS or PKCS12) |
— |
TLS - Truststore |
Path to the truststore that is used when connecting to LDAP using LDAPS or START_TLS |
— |
TLS - Truststore Password |
Password for the truststore that is used when connecting to LDAP using LDAPS or START_TLS |
— |
TLS - Truststore Type |
Type of the truststore that is used when connecting to LDAP using LDAPS or START_TLS (i.e. JKS or PKCS12) |
— |
TLS - Client Auth |
Client authentication policy when connecting to LDAP using LDAPS or START_TLS. Possible values are |
NONE |
TLS - Protocol |
Protocol to use when connecting to LDAP using LDAPS or START_TLS. (i.e. TLS, TLSv1.1, TLSv1.2, etc.) |
— |
TLS - Shutdown Gracefully |
Specifies whether the TLS should be shut down gracefully before the target context is closed |
False |
Referral Strategy |
Strategy for handling referrals |
FOLLOW |
Connect Timeout |
Duration of connect timeout |
10 sec |
Read Timeout |
Duration of read timeout |
10 sec |
LDAP URL |
Space-separated list of URLs of the LDAP servers (e.g. |
— |
User Search Base |
Base DN for searching for users (e.g. |
— |
User Search Filter |
Filter for searching for users against the User Search Base (e.g. |
— |
Identity Strategy |
Strategy to identify users. Possible values are |
USE_DN |
Authentication Expiration |
Duration of how long the user authentication is valid for. If the user never logs out, they will be required to log back in following this duration |
12 hours |
| Parameter | Description | Default value |
|---|---|---|
Authentication Strategy |
How the connection to the LDAP server is authenticated |
ANONYMOUS |
Manager DN |
DN of a user that has an entry in the Active Directory with right to search users and groups. Will be used to bind to an LDAP server to search for users |
— |
Manager Password |
Password of the manager that is used to bind to the LDAP server to search for users |
— |
TLS - Keystore |
Path to the keystore that is used when connecting to LDAP using LDAPS or START_TLS |
— |
TLS - Keystore Password |
Password for the keystore that is used when connecting to LDAP using LDAPS or START_TLS |
— |
TLS - Keystore Type |
Type of the keystore that is used when connecting to LDAP using LDAPS or START_TLS (i.e. JKS or PKCS12) |
— |
TLS - Truststore |
Path to the truststore that is used when connecting to LDAP using LDAPS or START_TLS |
— |
TLS - Truststore Password |
Password for the truststore that is used when connecting to LDAP using LDAPS or START_TLS |
— |
TLS - Truststore Type |
Type of the truststore that is used when connecting to LDAP using LDAPS or START_TLS (i.e. JKS or PKCS12) |
— |
TLS - Client Auth |
Client authentication policy when connecting to LDAP using LDAPS or START_TLS. Possible values are |
NONE |
TLS - Protocol |
Protocol to use when connecting to LDAP using LDAPS or START_TLS. (i.e. TLS, TLSv1.1, TLSv1.2, etc.) |
— |
TLS - Shutdown Gracefully |
Specifies whether the TLS should be shut down gracefully before the target context is closed |
— |
Referral Strategy |
Strategy for handling referrals |
FOLLOW |
Connect Timeout |
Duration of connect timeout |
10 sec |
Read Timeout |
Duration of read timeout |
10 sec |
LDAP URL |
Space-separated list of URLs of the LDAP servers (e.g. |
— |
Page Size |
Sets the page size when retrieving users and groups. If not specified, no paging is performed |
— |
Sync Interval |
Duration of time between syncing users and groups. Minimum allowable value is |
30 mins |
User Search Base |
Base DN for searching for users (e.g. |
— |
User Object Class |
Object class for identifying users (e.g. |
— |
User Search Scope |
Search scope for searching users |
ONE_LEVEL |
User Search Filter |
Filter for searching for users against the User Search Base (e.g. |
— |
User Identity Attribute |
Attribute to use to extract user identity (e.g. |
— |
User Group Name Attribute |
Attribute to use to define group membership (e.g. |
— |
User Group Name Attribute - Referenced Group Attribute |
If blank, the value of the attribute defined in User Group Name Attribute is expected to be the full DN of the group. If not blank, this property will define the attribute of the group LDAP entry that the value of the attribute defined in User Group Name Attribute is referencing (e.g. |
— |
Group Search Base |
Base DN for searching for groups (e.g. |
— |
Group Object Class |
Object class for identifying groups (e.g. |
— |
Group Search Scope |
Search scope for user group |
ONE_LEVEL |
Group Search Filter |
Filter for searching for groups against the Group Search Base. Optional |
— |
Group Name Attribute |
Attribute to use to extract group name (e.g. |
— |
Group Member Attribute |
Attribute to use to define group membership (e.g. |
— |
Group Member Attribute - Referenced User Attribute |
If blank, the value of the attribute defined in Group Member Attribute is expected to be the full DN of the user. If not blank, this property will define the attribute of the user LDAP entry that the value of the attribute defined in Group Member Attribute is referencing (e.g. |
— |
nifi.security.group.mapping.value.anygroup |
Replacement value for mapping NiFi Server user group name DNs to pattern |
$1 |
nifi.security.group.mapping.pattern.anygroup |
Pattern for mapping NiFi Server user group name DNs |
(?i)cn=([,]+),.* |
nifi.security.group.mapping.transform.anygroup |
Transformation applied to NiFi Server user group name DNs |
LOWER |
nifi.security.identity.mapping.value.dn |
The replacement value for matching NiFi user DN identifiers |
$1 |
nifi.security.identity.mapping.pattern.dn |
Pattern for mapping NiFi user DN identifiers |
^(.*)$ |
nifi.security.identity.mapping.transform.dn |
Transformation applied to DN identifiers of NiFi users |
NONE |
nifi.registry.security.group.mapping.value.anygroup |
Replacement value for mapping NiFi Registry user group name DNs to pattern |
$1 |
nifi.registry.security.group.mapping.pattern.anygroup |
Pattern for mapping NiFi Registry user group name DNs |
(?i)cn=([,]+),.* |
nifi.registry.security.group.mapping.transform.anygroup |
Transformation applied to NiFi Registry user group name DNs |
LOWER |
nifi.registry.security.identity.mapping.value.dn |
Replacement value for mapping NiFi Registry user DN identifiers |
$1 |
nifi.registry.security.identity.mapping.pattern.dn |
Pattern for mapping DN identifiers of NiFi Registry users |
^(.*)$ |
nifi.registry.security.identity.mapping.transform.dn |
Transformation applied to the DN identifiers of NiFi Registry users |
LOWER |
Analytics platform configurations
| Parameter | Description | Default value |
|---|---|---|
nifi.analytics.predict.interval |
Time interval in which analytic predictions should be made (e.g. queue saturation) |
3 mins |
nifi.analytics.query.interval |
Time interval to query for past observations (for example, the last 3 minutes of snapshots). The value must be at least 3 times greater than the specified value |
5 mins |
nifi.analytics.connection.model.implementation |
Implementation class for the state analysis model used for connection predictions |
Ordinary Least Squares |
nifi.analytics.connection.model.score.name |
Name of the scoring type to use to score the model |
rSquared |
nifi.analytics.connection.model.score.threshold |
Threshold for the scoring value (the score model must be above the specified threshold) |
.90 |
Parameters defining the place to install the NiFi service
| Parameter | Description | Default value |
|---|---|---|
NIFI_HOME |
Directory for NiFi installation |
/usr/lib/nifi-server |
NIFI_PID_DIR |
Directory to store the NiFi process ID |
/var/run/nifi |
NIFI_LOG_DIR |
Directory to store the logs |
/var/log/nifi |
NIFI_ALLOW_EXPLICIT_KEYTAB |
Defines whether to prevent of the old free-form keytab properties that were left around for backwards compatibility |
true |
Allows users to set NiFi startup parameters in the bootstrap.conf file using the Add key, value field.
Parameters for configuring the StandardHashiCorpVaultClientService controller service in accordance with the Sensitive Properties Provider HashiCorpVault.
| Parameter | Description | Default value |
|---|---|---|
vault.uri |
HTTP or HTTPS URI for HashiCorp Vault, required to enable the Sensitive Properties Provider |
— |
vault.authentication.properties.file |
Path to the optional authentication properties file for the Spring Vault environment. To use it, you need to specify the |
/etc/nifi/conf/vault-auth.properties |
vault.connection.timeout |
Connection timeout |
5 secs |
vault.read.timeout |
Read timeout |
15 secs |
Add key, value |
Parameters and their values entered in this field override the parameters specified in the ADCM user interface. This field also allows you to set values for all user parameters that are not displayed in the interface, but are allowed in the configuration file bootstrap-hashicorp-vault.conf |
— |
Authentication parameters of the Spring Vault configuration environment.
| Parameter | Description | Default value |
|---|---|---|
vault.token |
Vault authentication token |
— |
SSL enablement parameters for the StandardHashiCorpVaultClientService controller service.
| Parameter | Description | Default value |
|---|---|---|
vault.ssl.enabledCipherSuites |
Enabled SSL/TLS cipher suites, separated by commas |
— |
vault.ssl.enabledProtocols |
Enabled SSL/TLS protocols, separated by commas |
TLSv1.3 |
vault.ssl.key-store |
Path to the keystore file |
— |
vault.ssl.key-store-password |
Keystore password |
— |
vault.ssl.key-store-type |
Keystore type |
PKCS12 |
vault.ssl.trust-store |
Path to the truststore file |
— |
vault.ssl.trust-store-password |
Truststore password |
— |
vault.ssl.trust-store-type |
Truststore type |
PKCS12 |
Add key, value |
Parameters and their values entered in this field override the parameters specified in the ADCM user interface. This field also allows you to set values for all user parameters that are not displayed in the interface, but are allowed in the configuration file bootstrap-hashicorp-vault.conf |
— |
| Parameter | Description | Default value |
|---|---|---|
nifi.flow.configuration.file |
Location of the XML-based flow configuration file |
/etc/nifi/conf/flow.xml.gz |
nifi.flow.configuration.json.file |
Directory where the flow.json.gz file is stored, containing the entire flow for a given NiFi instance. Used to restore or migrate a flow |
/etc/nifi/conf/flow.json.gz |
nifi.flow.configuration.archive.enabled |
Enables NiFi to create a fallback schema for automatic stream updates |
true |
nifi.cluster.node.connection.timeout |
When connecting to another node in the cluster, specifies how long this node should wait before considering the connection a failure |
5 sec |
nifi.cluster.node.read.timeout |
When communicating with another node in the cluster, specifies how long this node should wait to receive information from the remote node before considering the communication with the node a failure |
5 sec |
nifi.zookeeper.connect.timeout |
How long to wait when connecting to ZooKeeper before considering the connection a failure |
3 secs |
nifi.zookeeper.session.timeout |
How long to wait after losing a connection to ZooKeeper before the session is expired |
3 secs |
nifi.variable.registry.properties |
Comma-separated list of file location paths for one or more custom property files |
/etc/nifi/conf/extra-args.properties |
nifi.remote.input.http.enabled |
Specifies whether HTTP Site-to-Site should be enabled on this host |
true |
nifi.remote.input.http.transaction.ttl |
Specifies how long a transaction can stay alive on the server |
30 sec |
nifi.remote.contents.cache.expiration |
Specifies how long NiFi should cache information about a remote NiFi instance when communicating via Site-to-Site |
30 secs |
nifi.flow.configuration.archive.max.time |
Lifespan of archived flow.xml files |
30 days |
nifi.flow.configuration.archive.max.storage |
Total data size allowed for the archived flow.xml files |
500 MB |
nifi.flow.configuration.archive.max.count |
Number of archive files allowed |
— |
nifi.flowcontroller.autoResumeState |
Indicates whether |
true |
nifi.flowcontroller.graceful.shutdown.period |
Indicates the shutdown period |
10 sec |
nifi.flowservice.writedelay.interval |
When many changes are made to the flow.xml, this property specifies how long to wait before writing out the changes, so as to batch the changes into a single write |
500 ms |
nifi.administrative.yield.duration |
If a component allows an unexpected exception to escape, it is considered a bug. As a result, the framework will pause (or administratively yield) the component for this amount of time |
30 sec |
nifi.bored.yield.duration |
When a component has no work to do (i.e. is bored), this is the amount of time it will wait before checking to see if it has new data to work on |
10 millis |
nifi.ui.banner.text |
Banner text that may be configured to display at the top of the User Interface |
— |
nifi.ui.autorefresh.interval |
Interval at which the User Interface auto-refreshes |
30 sec |
nifi.state.management.provider.local |
ID of the Local State Provider to use |
local-provider |
nifi.state.management.provider.cluster |
ID of the Cluster State Provider to use |
zk-provider |
nifi.state.management.embedded.zookeeper.start |
Specifies whether or not this instance of NiFi should start an embedded ZooKeeper Server |
false |
nifi.h2.url.append |
Specifies additional arguments to add to the connection string for the H2 database |
;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=FALSE |
nifi.flowfile.repository.implementation |
FlowFile Repository implementation. To store flowfiles in memory instead of on disk (accepting data loss in the event of power/machine failure or a restart of NiFi), set this property to |
org.apache.nifi.controller.repository.WriteAheadFlowFileRepository |
nifi.flowfile.repository.wal.implementation |
If the repository implementation is configured to use the WriteAheadFlowFileRepository, this property can be used to specify which implementation of the write-whead log should be used |
org.apache.nifi.wali.SequentialAccessWriteAheadLog |
nifi.flowfile.repository.partitions |
Number of partitions |
256 |
nifi.flowfile.repository.checkpoint.interval |
FlowFile Repository checkpoint interval |
2 mins |
nifi.flowfile.repository.always.sync |
If set to |
false |
nifi.swap.manager.implementation |
Swap Manager implementation |
org.apache.nifi.controller.FileSystemSwapManager |
nifi.queue.swap.threshold |
Queue threshold at which NiFi starts to swap FlowFile information to disk |
20000 |
nifi.swap.in.period |
Swap in period |
5 sec |
nifi.swap.in.threads |
Number of threads to use for swapping in |
1 |
nifi.swap.out.period |
Swap out period |
5 sec |
nifi.swap.out.threads |
Number of threads to use for swapping out |
4 |
nifi.content.repository.implementation |
Content Repository implementation. The default value is |
org.apache.nifi.controller.repository.FileSystemRepository |
nifi.content.claim.max.appendable.size |
Maximum size for a content claim |
1 MB |
nifi.content.claim.max.flow.files |
Maximum number of FlowFiles to assign to one content claim |
100 |
nifi.content.repository.archive.max.retention.period |
If archiving is enabled, then this property specifies the maximum amount of time to keep the archived data |
12 hours |
nifi.content.repository.archive.max.usage.percentage |
If archiving is enabled then this property must have a value that indicates the content repository disk usage percentage at which archived data begins to be removed. If the archive is empty and content repository disk usage is above this percentage, then archiving is temporarily disabled. Archiving will resume when disk usage is below this percentage |
50% |
nifi.content.repository.archive.enabled |
To enable content archiving, set this to |
true |
nifi.content.repository.always.sync |
If set to |
false |
nifi.content.viewer.url |
URL for a web-based content viewer if one is available |
../nifi-content-viewer/ |
nifi.provenance.repository.implementation |
Provenance Repository implementation. Possible values are:
|
org.apache.nifi.provenance.WriteAheadProvenanceRepository |
nifi.provenance.repository.debug.frequency |
Controls the number of events processed between |
1_000_000 |
nifi.provenance.repository.encryption.key.provider.implementation |
Fully-qualified class name of the key provider |
— |
nifi.provenance.repository.encryption.key.provider.location |
Path to the key definition resource |
— |
nifi.provenance.repository.encryption.key.id |
Active key ID to use for encryption (e.g. |
— |
nifi.provenance.repository.encryption.key |
Key to use for StaticKeyProvider |
— |
nifi.provenance.repository.max.storage.time |
Maximum amount of time to keep data provenance information |
24 hours |
nifi.provenance.repository.max.storage.size |
Maximum amount of data provenance information to store at a time |
1 GB |
nifi.provenance.repository.rollover.time |
Amount of time to wait before rolling over the latest data provenance information so that it is available in the User Interface |
30 secs |
nifi.provenance.repository.rollover.size |
Amount of information to roll over at a time |
100 MB |
nifi.provenance.repository.query.threads |
Number of threads to use for Provenance Repository queries |
2 |
nifi.provenance.repository.index.threads |
Number of threads to use for indexing Provenance events so that they are searchable |
2 |
nifi.provenance.repository.compress.on.rollover |
Indicates whether to compress the provenance information when rolling it over |
true |
nifi.provenance.repository.always.sync |
If set to |
false |
nifi.provenance.repository.indexed.fields |
Comma-separated list of the fields that should be indexed and made searchable |
EventType, FlowFileUUID, Filename, ProcessorID, Relationship |
nifi.provenance.repository.indexed.attributes |
Comma-separated list of FlowFile Attributes that should be indexed and made searchable |
— |
nifi.provenance.repository.index.shard.size |
Large values for the shard size will result in more Java heap usage when searching the Provenance Repository but should provide better performance |
500 MB |
nifi.provenance.repository.max.attribute.length |
Indicates the maximum length that a FlowFile attribute can be when retrieving a Provenance Event from the repository. If the length of any attribute exceeds this value, it will be truncated when the event is retrieved |
65536 |
nifi.provenance.repository.concurrent.merge.threads |
Specifies the maximum number of threads that are allowed to be used for each of the storage directories |
2 |
nifi.provenance.repository.buffer.size |
Provenance Repository buffer size |
100000 |
nifi.components.status.repository.implementation |
Component Status Repository implementation |
org.apache.nifi.controller.status.history.VolatileComponentStatusRepository |
nifi.components.status.repository.buffer.size |
Specifies the buffer size for the Component Status Repository |
1440 |
nifi.components.status.snapshot.frequency |
Indicates how often to present a snapshot of the components status history |
1 min |
nifi.web.war.directory |
Location of the web war directory |
./lib |
nifi.web.jetty.working.directory |
Location of the Jetty working directory |
./work/jetty |
nifi.web.jetty.threads |
Number of Jetty threads |
200 |
nifi.web.max.header.size |
Maximum size allowed for request and response headers |
16 KB |
nifi.web.proxy.context.path |
Comma-separated list of allowed HTTP X-ProxyContextPath or X-Forwarded-Context header values to consider. By default, this value is blank meaning all requests containing a proxy context path are rejected |
— |
nifi.web.proxy.host |
Comma-separated list of allowed HTTP Host header values to consider when NiFi is running securely and will be receiving requests to a different host[:port] than it is bound to. For example, when running in a Docker container or behind a proxy (e.g. |
— |
nifi.sensitive.props.key |
Password (source string) from which to extract the encryption key for the algorithm specified in the |
mysensetivekey |
nifi.sensitive.props.key.protected |
Protected password (source string) used to obtain the encryption key for the algorithm specified in the |
— |
nifi.sensitive.props.algorithm |
Algorithm used to encrypt sensitive properties |
NIFI_PBKDF2_AES_GCM_256 |
nifi.sensitive.props.provider |
Sensitive property provider |
BC |
nifi.sensitive.props.additional.keys |
Comma-separated list of properties to encrypt in addition to the default sensitive properties |
— |
nifi.security.user.authorizer |
Specifies which of the configured Authorizers in the authorizers.xml file to use. By default, it is set to file-provider |
managed-authorizer |
nifi.security.ocsp.responder.url |
URL for the Online Certificate Status Protocol (OCSP) responder if one is being used |
— |
nifi.security.ocsp.responder.certificate |
Location of the OCSP responder certificate if one is being used. It is blank by default |
— |
nifi.security.user.oidc.discovery.url |
Discovery URL for the desired OpenId Connect Provider |
— |
nifi.security.user.oidc.connect.timeout |
Connect timeout when communicating with the OpenId Connect Provider |
5 secs |
nifi.security.user.oidc.read.timeout |
Read timeout when communicating with the OpenId Connect Provider |
5 secs |
nifi.security.user.oidc.client.id |
Client id for NiFi after registration with the OpenId Connect Provider |
— |
nifi.security.user.oidc.client.secret |
Client secret for NiFi after registration with the OpenId Connect Provider |
— |
nifi.security.user.oidc.preferred.jwsalgorithm |
Preferred algorithm for validating identity tokens. If this value is blank, it will default to |
— |
nifi.security.user.knox.url |
URL for the Apache Knox login page |
— |
nifi.security.user.knox.publicKey |
Path to the Apache Knox public key that will be used to verify the signatures of the authentication tokens in the HTTP Cookie |
— |
nifi.security.user.knox.cookieName |
Name of the HTTP Cookie that Apache Knox will generate after successful login |
hadoop-jwt |
nifi.security.user.knox.audiences |
Optional. A comma-separated list of allowed audiences. If set, the audience in the token must be present in this listing. The audience that is populated in the token can be configured in Knox |
— |
nifi.cluster.protocol.heartbeat.interval |
Interval at which nodes should emit heartbeats to the Cluster Coordinator |
5 sec |
nifi.cluster.node.protocol.port |
Node’s protocol port |
11433 |
nifi.cluster.node.protocol.threads |
Number of threads that should be used to communicate with other nodes in the cluster |
10 |
nifi.cluster.node.protocol.max.threads |
Maximum number of threads that should be used to communicate with other nodes in the cluster |
50 |
nifi.cluster.node.event.history.size |
When the state of a node in the cluster is changed, an event is generated and can be viewed on the Cluster page. This value indicates how many events to keep in memory for each node |
25 |
nifi.cluster.node.max.concurrent.requests |
Maximum number of outstanding web requests that can be replicated to nodes in the cluster. If this number of requests is exceeded, the embedded Jetty server will return a |
100 |
nifi.cluster.firewall.file |
Location of the node firewall file. This is a file that may be used to list all the nodes that are allowed to connect to the cluster. It provides an additional layer of security. This value is blank by default, meaning that no firewall file is to be used |
— |
nifi.cluster.flow.election.max.wait.time |
Specifies the amount of time to wait before electing a Flow as the "correct" Flow. If the number of Nodes that have voted is equal to the number specified by the |
5 mins |
nifi.cluster.load.balance.host |
Specifies the hostname to listen on for incoming connections for load balancing data across the cluster. If not specified, will default to the value used by the |
— |
nifi.cluster.load.balance.port |
Specifies the port to listen on for incoming connections for load balancing data across the cluster |
6342 |
nifi.cluster.load.balance.connections.per.node |
Maximum number of connections to create between this node and each other node in the cluster. For example, if there are 5 nodes in the cluster and this value is set to 4, there will be up to 20 socket connections established for load-balancing purposes (5 x 4 = 20) |
4 |
nifi.cluster.load.balance.max.thread.count |
Maximum number of threads to use for transferring data from this node to other nodes in the cluster. While a given thread can only write to a single socket at a time, a single thread is capable of servicing multiple connections simultaneously because a given connection may not be available for reading/writing at any given time |
8 |
nifi.cluster.load.balance.comms.timeout |
When communicating with another node, if this amount of time elapses without making any progress when reading from or writing to a socket, then a TimeoutException will be thrown. This will then result in the data either being retried or sent to another node in the cluster, depending on the configured Load Balancing Strategy |
30 sec |
nifi.remote.input.socket.port |
Remote input socket port for Site-to-Site communication |
10443 |
nifi.remote.input.secure |
This indicates whether communication between this instance of NiFi and remote NiFi instances should be secure |
false |
nifi.security.keystore |
Full path and name of the keystore |
/tmp/keystore.jks |
nifi.security.keystoreType |
Keystore type |
JKS |
nifi.security.keystorePasswd |
Keystore password |
— |
nifi.security.keyPasswd |
Key password |
— |
nifi.security.truststore |
Full path and name of the truststore |
— |
nifi.security.truststoreType |
Truststore type |
JKS |
nifi.security.truststorePasswd |
Truststore password |
— |
Add key, value |
Parameters and their values entered in this field override the parameters specified in the ADCM user interface. This field also allows you to set values for all user parameters that are not displayed in the interface, but are allowed in the configuration file nifi.properties |
— |
File for specifying additional arguments (extra-args) using the Add property field.
Setting logging levels and log rotate for NiFi Server
| Parameter | Description | Default value |
|---|---|---|
app_file_max_history |
Maximum number of files for applications |
10 |
user_file_max_history |
Maximum user files |
10 |
boot_file_max_history |
Maximum number of files for Boot |
5 |
root_level |
Event level |
INFO |
Setting the structure of the logging configuration file for NiFi Server
| Logger | Default package names | Default event level |
|---|---|---|
app_loggers |
org.wali |
WARN |
org.apache.nifi |
INFO |
|
org.eclipse.jetty |
INFO |
|
org.apache.nifi.cluster |
INFO |
|
org.apache.nifi.processors |
WARN |
|
org.apache.nifi.server.JettyServer |
INFO |
|
org.apache.nifi.processors.standard.LogMessage |
INFO |
|
org.apache.nifi.processors.standard.LogAttribute |
INFO |
|
org.apache.nifi.controller.repository.StandardProcessSession |
WARN |
|
user_events_loggers |
org.apache.nifi.web.security |
INFO |
org.apache.nifi.web.api.config |
INFO |
|
org.apache.nifi.authorization |
INFO |
|
org.apache.nifi.cluster.authorization |
INFO |
|
org.apache.nifi.web.filter.RequestLogger |
INFO |
|
bootstrap_loggers |
org.apache.nifi.bootstrap |
INFO |
org.apache.nifi.bootstrap.Command |
INFO |
|
org.apache.nifi.StdOut |
INFO |
|
org.apache.nifi.StdErr |
INFO |
|
custom_logger |
— |
— |
Template for customizing the NiFi logback.xml file.
Default value:
<?xml version="1.0" encoding="UTF-8"?>
<!--
Maintained by ADCM
-->
{% set logback = services.nifi.config['nifi_logback_content'] %}
<configuration scan="true" scanPeriod="30 seconds">
<contextListener class="ch.qos.logback.classic.jul.LevelChangePropagator">
<resetJUL>true</resetJUL>
</contextListener>
<appender name="APP_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${org.apache.nifi.bootstrap.config.log.dir}/nifi-app.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<!--
For daily rollover, use 'app_%d.log'.
For hourly rollover, use 'app_%d{yyyy-MM-dd_HH}.log'.
To GZIP rolled files, replace '.log' with '.log.gz'.
To ZIP rolled files, replace '.log' with '.log.zip'.
-->
<fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/nifi-app_%d{yyyy-MM-dd_HH}.%i.log</fileNamePattern>
<maxFileSize>100MB</maxFileSize>
<!-- keep 30 log files worth of history -->
<maxHistory>{{ logback.app_file_max_history }}</maxHistory>
</rollingPolicy>
<immediateFlush>true</immediateFlush>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder>
</appender>
<appender name="USER_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${org.apache.nifi.bootstrap.config.log.dir}/nifi-user.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!--
For daily rollover, use 'user_%d.log'.
For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'.
To GZIP rolled files, replace '.log' with '.log.gz'.
To ZIP rolled files, replace '.log' with '.log.zip'.
-->
<fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/nifi-user_%d.log</fileNamePattern>
<!-- keep 30 log files worth of history -->
<maxHistory>{{ logback.user_file_max_history }}</maxHistory>
</rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder>
</appender>
<appender name="BOOTSTRAP_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${org.apache.nifi.bootstrap.config.log.dir}/nifi-bootstrap.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!--
For daily rollover, use 'user_%d.log'.
For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'.
To GZIP rolled files, replace '.log' with '.log.gz'.
To ZIP rolled files, replace '.log' with '.log.zip'.
-->
<fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/nifi-bootstrap_%d.log</fileNamePattern>
<!-- keep 5 log files worth of history -->
<maxHistory>{{ logback.boot_file_max_history }}</maxHistory>
</rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder>
</appender>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder>
</appender>
<!-- valid logging levels: TRACE, DEBUG, INFO, WARN, ERROR -->
{% for key, value in logback.app_loggers | dictsort -%}
<logger name="{{ key }}" level="{{ value }}"/>
{% endfor -%}
<logger name="org.apache.zookeeper.ClientCnxn" level="ERROR" />
<logger name="org.apache.zookeeper.server.NIOServerCnxn" level="ERROR" />
<logger name="org.apache.zookeeper.server.NIOServerCnxnFactory" level="ERROR" />
<logger name="org.apache.zookeeper.server.quorum" level="ERROR" />
<logger name="org.apache.zookeeper.ZooKeeper" level="ERROR" />
<logger name="org.apache.zookeeper.server.PrepRequestProcessor" level="ERROR" />
<logger name="org.apache.calcite.runtime.CalciteException" level="OFF" />
<logger name="org.apache.curator.framework.recipes.leader.LeaderSelector" level="OFF" />
<logger name="org.apache.curator.ConnectionState" level="OFF" />
<!-- Suppress non-error messages due to excessive logging by class or library -->
<logger name="com.sun.jersey.spi.container.servlet.WebComponent" level="ERROR"/>
<logger name="com.sun.jersey.spi.spring" level="ERROR"/>
<logger name="org.springframework" level="ERROR"/>
<!-- Suppress non-error messages due to known warning about redundant path annotation (NIFI-574) -->
<logger name="com.sun.jersey.spi.inject.Errors" level="ERROR"/>
<logger name="org.glassfish.jersey.internal.Errors" level="ERROR"/>
<!-- Suppress non-error messages due to Jetty AnnotationParser emitting a large amount of WARNS. Issue described in NIFI-5479. -->
<logger name="org.eclipse.jetty.annotations.AnnotationParser" level="ERROR"/>
<!--
Logger for capturing user events. We do not want to propagate these
log events to the root logger. These messages are only sent to the
user-log appender.
-->
{% for key, value in logback.user_events_loggers | dictsort -%}
<logger name="{{ key }}" level="{{ value }}" additivity="false">
<appender-ref ref="USER_FILE"/>
</logger>
{% endfor -%}
<!--
Logger for capturing Bootstrap logs and NiFi's standard error and standard out.
-->
{% for key, value in logback.bootstrap_loggers | dictsort -%}
<logger name="{{ key }}" level="{{ value }}" additivity="false">
<appender-ref ref="BOOTSTRAP_FILE"/>
{% if key == "org.apache.nifi.bootstrap.Command" %}
<appender-ref ref="CONSOLE" />
{% endif -%}
</logger>
{% endfor -%}
<!--
Custom Logger
-->
{% if logback.custom_logger is not none -%}
{% if logback.custom_logger | length > 0 -%}
{% for key, value in logback.custom_logger | dictsort -%}
<logger name="{{ key }}" level="{{ value }}"/>
{% endfor -%}
{% endif -%}
{% endif -%}
<root level="{{ logback.root_level }}">
<appender-ref ref="APP_FILE"/>
</root>
</configuration>
Template for customizing the NiFi state-management.xml file.
Default value:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<!--
Maintained by ADCM
-->
{%- if cluster.config.cluster_znode is defined and cluster.config.cluster_znode is not none %}
{% set zookeeper_connect = cluster.config.cluster_znode.split('/')[0] %}
{%- endif -%}
<stateManagement>
<!--
State Provider that stores state locally in a configurable directory. This Provider requires the following properties:
Directory - the directory to store components' state in. If the directory being used is a sub-directory of the NiFi installation, it
is important that the directory be copied over to the new version when upgrading NiFi.
Always Sync - If set to true, any change to the repository will be synchronized to the disk, meaning that NiFi will ask the operating system not to cache the information. This is very
expensive and can significantly reduce NiFi performance. However, if it is false, there could be the potential for data loss if either there is a sudden power loss or the
operating system crashes. The default value is false.
Partitions - The number of partitions.
Checkpoint Interval - The amount of time between checkpoints.
-->
<local-provider>
<id>local-provider</id>
<class>org.apache.nifi.controller.state.providers.local.WriteAheadLocalStateProvider</class>
<property name="Directory">{{ nifi_home }}/conf/state/local</property>
<property name="Always Sync">false</property>
<property name="Partitions">16</property>
<property name="Checkpoint Interval">2 mins</property>
</local-provider>
<!--
State Provider that is used to store state in ZooKeeper. This Provider requires the following properties:
Root Node - the root node in ZooKeeper where state should be stored. The default is '/nifi', but it is advisable to change this to a different value if not using
the embedded ZooKeeper server and if multiple NiFi instances may all be using the same ZooKeeper Server.
Connect String - A comma-separated list of host:port pairs to connect to ZooKeeper. For example, myhost.mydomain:2181,host2.mydomain:5555,host3:6666
Session Timeout - Specifies how long this instance of NiFi is allowed to be disconnected from ZooKeeper before creating a new ZooKeeper Session. Default value is "30 seconds"
Access Control - Specifies which Access Controls will be applied to the ZooKeeper ZNodes that are created by this State Provider. This value must be set to one of:
- Open : ZNodes will be open to any ZooKeeper client.
- CreatorOnly : ZNodes will be accessible only by the creator. The creator will have full access to create children, read, write, delete, and administer the ZNodes.
This option is available only if access to ZooKeeper is secured via Kerberos or if a Username and Password are set.
-->
<cluster-provider>
<id>zk-provider</id>
<class>org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider</class>
<property name="Connect String">{{ zookeeper_connect | default('') }}</property>
<property name="Root Node">/arenadata/cluster/{{ cluster.id }}/nifi</property>
<property name="Session Timeout">10 seconds</property>
<property name="Access Control">Open</property>
</cluster-provider>
<!--
Cluster State Provider that stores state in Redis. This can be used as an alternative to the ZooKeeper State Provider.
This provider requires the following properties:
Redis Mode - The type of Redis instance:
- Standalone
- Sentinel
- Cluster (currently not supported for state-management due to use of WATCH command which Redis does not support in clustered mode)
Connection String - The connection string for Redis.
- In a standalone instance this value will be of the form hostname:port.
- In a sentinel instance this value will be the comma-separated list of sentinels, such as host1:port1,host2:port2,host3:port3.
- In a clustered instance this value will be the comma-separated list of cluster masters, such as host1:port,host2:port,host3:port.
This provider has the following optional properties:
Key Prefix - The prefix for each key stored by this state provider. When sharing a single Redis across multiple NiFi instances, setting a unique
value for the Key Prefix will make it easier to identify which instances the keys came from (default nifi/components/).
Database Index - The database index to be used by connections created from this connection pool.
See the databases property in redis.conf, by default databases 0-15 will be available.
Communication Timeout - The timeout to use when attempting to communicate with Redis.
Cluster Max Redirects - The maximum number of redirects that can be performed when clustered.
Sentinel Master - The name of the sentinel master, require when Mode is set to Sentinel.
Password - The password used to authenticate to the Redis server. See the requirepass property in redis.conf.
Pool - Max Total - The maximum number of connections that can be allocated by the pool (checked out to clients, or idle awaiting checkout).
A negative value indicates that there is no limit.
Pool - Max Idle - The maximum number of idle connections that can be held in the pool, or a negative value if there is no limit.
Pool - Min Idle - The target for the minimum number of idle connections to maintain in the pool. If the configured value of Min Idle is
greater than the configured value for Max Idle, then the value of Max Idle will be used instead.
Pool - Block When Exhausted - Whether or not clients should block and wait when trying to obtain a connection from the pool when the pool
has no available connections. Setting this to false means an error will occur immediately when a client requests a connection and
none are available.
Pool - Max Wait Time - The amount of time to wait for an available connection when Block When Exhausted is set to true.
Pool - Min Evictable Idle Time - The minimum amount of time an object may sit idle in the pool before it is eligible for eviction.
Pool - Time Between Eviction Runs - The amount of time between attempting to evict idle connections from the pool.
Pool - Num Tests Per Eviction Run - The number of connections to tests per eviction attempt. A negative value indicates to test all connections.
Pool - Test On Create - Whether or not connections should be tested upon creation (default false).
Pool - Test On Borrow - Whether or not connections should be tested upon borrowing from the pool (default false).
Pool - Test On Return - Whether or not connections should be tested upon returning to the pool (default false).
Pool - Test While Idle - Whether or not connections should be tested while idle (default true).
<cluster-provider>
<id>redis-provider</id>
<class>org.apache.nifi.redis.state.RedisStateProvider</class>
<property name="Redis Mode">Standalone</property>
<property name="Connection String">localhost:6379</property>
</cluster-provider>
-->
</stateManagement>
Template for customizing the NiFi bootstrap-notification-services.xml file.
Default value:
<?xml version="1.0"?>
<!--
Maintained by ADCM
-->
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<services>
<!-- This file is used to define how interested parties are notified when events in NiFi's lifecycle occur. -->
<!-- The format of this file is:
<services>
<service>
<id>service-identifier</id>
<class>org.apache.nifi.notifications.DesiredNotificationService</class>
<property name="property name">property value</property>
<property name="another property">another property value</property>
</service>
</services>
This file can contain 0 to many different service definitions.
The id can then be referenced from the bootstrap.conf file in order to configure the notification service
to be used when particular lifecycle events occur.
-->
<!--
<service>
<id>email-notification</id>
<class>org.apache.nifi.bootstrap.notification.email.EmailNotificationService</class>
<property name="SMTP Hostname"></property>
<property name="SMTP Port"></property>
<property name="SMTP Username"></property>
<property name="SMTP Password"></property>
<property name="SMTP TLS"></property>
<property name="From"></property>
<property name="To"></property>
</service>
-->
<!--
<service>
<id>http-notification</id>
<class>org.apache.nifi.bootstrap.notification.http.HttpNotificationService</class>
<property name="URL"></property>
</service>
-->
</services>
Allows users to set NiFi Registry startup parameters in the bootstrap.conf file using the Add key, value field.
| Parameter | Description | Default value |
|---|---|---|
nifi.registry.web.war.directory |
Location of the web war directory |
./lib |
nifi.registry.web.jetty.working.directory |
Location of the Jetty working directory |
./work/jetty |
nifi.registry.web.jetty.threads |
Number of the Jetty threads |
200 |
nifi.registry.security.needClientAuth |
Specifies that connecting clients must authenticate with a client certificate |
false |
nifi.registry.db.directory |
Location of the Registry database directory |
— |
nifi.registry.db.url.append |
Specifies additional arguments to add to the connection string for the Registry database |
— |
nifi.registry.db.url |
Full JBDC connection string |
jdbc:h2:/usr/lib/nifi-registry/database/nifi-registry-primary;AUTOCOMMIT=OFF;DB_CLOSE_ON_EXIT=FALSE;LOCK_MODE=3;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=FALSE |
nifi.registry.db.driver.class |
Class name of the JBDC driver |
org.h2.Driver |
nifi.registry.db.driver.directory |
Optional directory containing one or more JARs to add to the classpath |
— |
nifi.registry.db.username |
Username for the database |
nifireg |
nifi.registry.db.password |
Password for the database |
— |
nifi.registry.db.maxConnections |
Maximum number of connections for the connection pool |
5 |
nifi.registry.db.sql.debug |
Whether or not to enable debug logging for SQL statements |
false |
nifi.registry.sensitive.props.additional.keys |
Comma-separated list of properties for encryption in addition to the default sensitive properties |
nifi.registry.db.password |
nifi.registry.security.keystore |
Full path and the name of the keystore |
— |
nifi.registry.security.keystoreType |
Keystore type |
— |
nifi.registry.security.keystorePasswd |
Keystore password |
— |
nifi.registry.security.keyPasswd |
Key password |
— |
nifi.registry.security.truststore |
Full path and name of the truststore |
— |
nifi.registry.security.truststoreType |
Truststore type |
— |
nifi.registry.security.truststorePasswd |
Truststore password |
— |
| Parameter | Description | Default value |
|---|---|---|
jmx_port |
Port to which JMX metrics are sent |
9995 |
JMX Exporter Port |
Port for connecting to Prometheus JMX Exporter |
11206 |
Enables authentication for JMX in the service (used when access to the JMX port needs to be protected).
| Parameter | Description | Default value |
|---|---|---|
Username |
Username for authentication in JMX |
monitoring |
Password |
User password for authentication in JMX |
— |
| Parameter | Description | Default value |
|---|---|---|
Filesystem Flow Storage Directory |
Filesystem flow storage directory |
/usr/lib/nifi-registry/flow_storage |
| Parameter | Description | Default value |
|---|---|---|
Git Flow Storage Directory |
File system path for a directory where flow contents files are persisted to. The directory must exist when NiFi Registry starts. It also must be initialized as a Git directory |
/usr/lib/nifi-registry/git_flow_storage |
Remote To Push |
When a new flow snapshot is created, this persistence provider updates files in the specified Git directory, then creates a commit to the local repository. If Remote To Push is defined, provider pushes to the specified remote repository (e.g. origin). To define more detailed remote spec such as branch names, use Refspec |
— |
Remote Access User |
Username is used to make push requests to the remote repository when Remote To Push is enabled, and the remote repository is accessed by HTTP protocol. If SSH is used, user authentications are done with SSH keys |
— |
Remote Access Password |
Password for the Remote Access User |
— |
Remote Clone Repository |
Remote repository URI to use to clone into Flow Storage Directory, if local repository is not present in Flow Storage Directory. If left empty, the Git directory needs to be configured as per initialaze Git directory. If URI is provided, then Remote Access User and Remote Access Password also should be present. Currently, default branch of remote wil be cloned |
— |
| Parameter | Description | Default value |
|---|---|---|
Extension Bundle Storage Directory |
Extension bundle storage directory |
/usr/lib/nifi-registry/extension_bundles |
| Parameter | Description | Default value |
|---|---|---|
S3 Credentials Provider |
Indicates how AWS credentials are provided |
STATIC |
S3 Region |
AWS region where the target S3 bucket exists |
— |
S3 Bucket Name |
Name of an existing S3 bucket to store extension bundles |
— |
S3 Key Prefix |
Optional prefix to prepend to S3 keys |
— |
S3 Access Key |
Access key for the STATIC credential provider |
— |
S3 Secret Access Key |
Secret access key for the STATIC credential provider |
— |
S3 Endpoint URL |
Optional override for the AWS S3 endpoint (e.g. for compatible storage) |
— |
Parameters that determine the location for installing the NiFi Registry component
| Parameter | Description | Default value |
|---|---|---|
NIFI_REGISTRY_HOME |
Directory for installing |
/usr/lib/nifi-registry |
NIFI_REGISTRY_PID_DIR |
Directory to store the NiFi Registry |
/var/run/nifi-registry |
NIFI_REGISTRY_LOG_DIR |
Directory to store the logs |
/var/log/nifi-registry |
Setting logging levels and log rotate for NiFi Registry
| Parameter | Description | Default value |
|---|---|---|
app_file_max_history |
Maximum number of files for applications |
10 |
events_file_max_history |
Maximum number of files for events |
5 |
boot_file_max_history |
Maximum number of files for Boot |
5 |
root_level |
Event level |
INFO |
Setting the structure of the logging configuration file for NiFi Server
| Logger | Default package names | Default event level |
|---|---|---|
app_loggers |
org.apache.nifi.registry |
INFO |
org.hibernate.SQL |
WARN |
|
org.hibernate.type |
INFO |
|
events_loggers |
org.apache.nifi.registry.provider.hook.LoggingEventHookProvider |
INFO |
bootstrap_loggers |
org.apache.nifi.registry.bootstrap |
INFO |
org.apache.nifi.registry.bootstrap.Command |
INFO |
|
org.apache.nifi.registry.StdOut |
INFO |
|
org.apache.nifi.registry.StdErr |
ERROR |
Template for customizing the NiFi Registry logback.xml file.
Default value:
<?xml version="1.0" encoding="UTF-8"?>
<!--
Maintained by ADCM
-->
{% set logback = services.nifi.config['nifi_registry_logback_content'] %}
<configuration scan="true" scanPeriod="30 seconds">
<contextListener class="ch.qos.logback.classic.jul.LevelChangePropagator">
<resetJUL>true</resetJUL>
</contextListener>
<appender name="APP_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${org.apache.nifi.registry.bootstrap.config.log.dir}/nifi-registry-app.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!--
For daily rollover, use 'app_%d.log'.
For hourly rollover, use 'app_%d{yyyy-MM-dd_HH}.log'.
To GZIP rolled files, replace '.log' with '.log.gz'.
To ZIP rolled files, replace '.log' with '.log.zip'.
-->
<fileNamePattern>${org.apache.nifi.registry.bootstrap.config.log.dir}/nifi-registry-app_%d{yyyy-MM-dd_HH}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<!-- keep 30 log files worth of history -->
<maxHistory>{{ logback.app_file_max_history }}</maxHistory>
</rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
<immediateFlush>true</immediateFlush>
</encoder>
</appender>
<appender name="BOOTSTRAP_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${org.apache.nifi.registry.bootstrap.config.log.dir}/nifi-registry-bootstrap.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!--
For daily rollover, use 'user_%d.log'.
For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'.
To GZIP rolled files, replace '.log' with '.log.gz'.
To ZIP rolled files, replace '.log' with '.log.zip'.
-->
<fileNamePattern>${org.apache.nifi.registry.bootstrap.config.log.dir}/nifi-registry-bootstrap_%d.log</fileNamePattern>
<!-- keep 5 log files worth of history -->
<maxHistory>{{ logback.boot_file_max_history }}</maxHistory>
</rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder>
</appender>
<appender name="EVENTS_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${org.apache.nifi.registry.bootstrap.config.log.dir}/nifi-registry-event.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!--
For daily rollover, use 'user_%d.log'.
For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'.
To GZIP rolled files, replace '.log' with '.log.gz'.
To ZIP rolled files, replace '.log' with '.log.zip'.
-->
<fileNamePattern>${org.apache.nifi.registry.bootstrap.config.log.dir}/nifi-registry-event_%d.log</fileNamePattern>
<!-- keep 5 log files worth of history -->
<maxHistory>{{ logback.events_file_max_history }}</maxHistory>
</rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date ## %msg%n</pattern>
</encoder>
</appender>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder>
</appender>
<!-- valid logging levels: TRACE, DEBUG, INFO, WARN, ERROR -->
{% for key, value in logback.app_loggers | dictsort -%}
<logger name="{{ key }}" level="{{ value }}"/>
{% endfor -%}
<!--
Logger for capturing Bootstrap logs and NiFi Registry's standard error and standard out.
-->
{% for key, value in logback.bootstrap_loggers | dictsort -%}
<logger name="{{ key }}" level="{{ value }}" additivity="false">
<appender-ref ref="BOOTSTRAP_FILE"/>
{% if key == "org.apache.nifi.registry.bootstrap.Command" %}
<appender-ref ref="CONSOLE" />
{% endif -%}
</logger>
{% endfor -%}
<!-- This will log all events to a separate file when the LoggingEventHookProvider is enabled in providers.xml -->
{% for key, value in logback.events_loggers | dictsort -%}
<logger name="{{ key }}" level="{{ value }}" additivity="false">
<appender-ref ref="EVENTS_FILE"/>
</logger>
{% endfor -%}
<root level="{{ logback.root_level }}">
<appender-ref ref="APP_FILE"/>
</root>
</configuration>
Template for customizing the NiFi Registry providers.xml file.
Default value:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<!--
Maintained by ADCM
-->
<providers>
{% if services.nifi.config['registry_flow_provider'] == 'FileSystemFlowPersistenceProvider' %}
{% set provider = services.nifi.config['registry_filesystem_flow_provider'] %}
<flowPersistenceProvider>
<class>org.apache.nifi.registry.provider.flow.FileSystemFlowPersistenceProvider</class>
<property name="Flow Storage Directory">{{ provider.flow_persistence_directory }}</property>
</flowPersistenceProvider>
{% elif services.nifi.config['registry_flow_provider'] == 'GitFlowPersistenceProvider' %}
{% set provider = services.nifi.config['registry_git_flow_provider'] %}
<flowPersistenceProvider>
<class>org.apache.nifi.registry.provider.flow.git.GitFlowPersistenceProvider</class>
<property name="Flow Storage Directory">{{ provider.flow_persistence_directory }}</property>
<property name="Remote To Push">{{ provider.remote_to_push }}</property>
<property name="Remote Access User">{{ provider.remote_access_user }}</property>
<property name="Remote Access Password">{{ provider.remote_access_password }}</property>
<property name="Remote Clone Repository">{{ provider.remote_clone_repository }}</property>
</flowPersistenceProvider>
{% endif %}
<!--
<eventHookProvider>
<class>org.apache.nifi.registry.provider.hook.ScriptEventHookProvider</class>
<property name="Script Path"></property>
<property name="Working Directory"></property>
-->
<!-- Optional Whitelist Event types
<property name="Whitelisted Event Type 1">CREATE_FLOW</property>
<property name="Whitelisted Event Type 2">DELETE_FLOW</property>
-->
<!--
</eventHookProvider>
-->
<!-- This will log all events to a separate file specified by the EVENT_APPENDER in logback.xml -->
<!--
<eventHookProvider>
<class>org.apache.nifi.registry.provider.hook.LoggingEventHookProvider</class>
</eventHookProvider>
-->
{% if services.nifi.config['registry_bundle_provider'] == 'FileSystemBundlePersistenceProvider' %}
{% set provider = services.nifi.config['registry_filesystem_bundle_provider'] %}
<extensionBundlePersistenceProvider>
<class>org.apache.nifi.registry.provider.extension.FileSystemBundlePersistenceProvider</class>
<property name="Extension Bundle Storage Directory">{{ provider.bundle_persistence_directory }}</property>
</extensionBundlePersistenceProvider>
{% endif %}
</providers>
Template for customizing the NiFi Registry registry-aliases.xml file.
Default value:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<!--
Maintained by ADCM
-->
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<aliases>
<!--
<alias>
<internal>LOCAL_NIFI_REGISTRY</internal>
<external>http://registry.nifi.apache.org:18080</external>
</alias>
-->
</aliases>
The user file template jaas.conf is intended for specifying user data for connecting clients of other services to the current service (paths to keytab files, the useTicketCache parameter, and others). For more information, see Configure a custom jaas.conf.
Default value:
{% if cluster.config.kerberos_client and cluster.config.kerberos_client.enable_kerberos %}
Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
storeKey=true
useTicketCache=false
principal="nifi/{{ ansible_fqdn }}@{{ cluster.config.kerberos_client.realm }}"
keyTab="{{ cluster.config.kerberos_client.keytab_dir }}/nifi.service.keytab";
};
{% endif %}