Prototype definition guide
This article describes the prototypes and their properties that you can use to describe services for integration with ADCM, particularly that are specified in configuration files for creating the ADCM bundles. The format of configuration (config) file should be YAML.
|
NOTE
The config file should be named config.yaml or config.yml. Multiple config.yaml files in a bundle are allowed. The name of the config file is reserved; other files should not be called by that name.
|
ADCM objects are created from prototypes. A prototype is a description of a single ADCM object.
Every prototype has the mandatory properties described below.
type
Every prototype has one of the following types:
-
cluster -
service -
provider(stands for hostprovider) -
host
version
version is a version of the prototype.
So the minimal working prototype should look like this:
---
- type: cluster
name: control
version: 1
A prototype also might have the optional properties described below.
actions
An action is an entity that can be used to describe an action on an ADCM object. Such an action is ultimately an Ansible playbook run with certain parameters under certain conditions.
You can describe special actions to bind their invocation to specific endpoints and UI buttons. You can choose from one of the four special actions names:
-
adcm_turn_on_maintenance_mode -
adcm_turn_off_maintenance_mode -
adcm_host_turn_on_maintenance_mode -
adcm_host_turn_off_maintenance_mode
Special actions are specified as follows:
actions:
adcm_turn_on_maintenance_mode
Special actions should be described in the matching prototypes.
| Name | Type of the prototype | Description |
|---|---|---|
adcm_turn_on_maintenance_mode |
Service/component |
Turns on maintenance mode on an object via the corresponding UI element |
adcm_turn_off_maintenance_mode |
Service/component |
Turns off maintenance mode on an object via the corresponding UI element |
adcm_host_turn_on_maintenance_mode |
Cluster |
Turns on maintenance mode on a cluster host via the corresponding UI element |
adcm_host_turn_off_maintenance_mode |
Cluster |
Turns off maintenance mode on a cluster host via the corresponding UI element |
adcm_delete_service |
Service |
Deletes service via the corresponding UI element |
Special actions that are described in a cluster prototype (adcm_host_turn_on_maintenance_mode, adcm_host_turn_off_maintenance_mode) should be host actions (the value of the host_action parameter should be true). Special actions are not supported for config, hc_acl, and ui_options.
Every prototype might have actions. Actions should be described as follows:
---
- type: cluster
name: control
version: 1
description: "Monitoring and Control Software"
actions:
install:
display_name: "Install Monitor Server"
description: |
By click on this button you install monitoring and controlling server ...
type: job
script_type: ansible
script: ansible/site.yaml
allow_to_terminate: true
params:
ansible_tags: install
jinja2_native: true
states:
available:
- created
on_success: installed
on_fail: created
config:
quorum:
type: integer
log_files:
- check
The install action from the example above has a job type and its script property specifies an Ansible playbook that resides in the <bundle_root>/ansible/site.yaml directory.
You can use a special dot syntax to specify a path to a script file. In this case, a script file will be searched in the same directory as config.yaml.
script: ./site.yaml
name is a short name that is unique for the given prototype.
display_name is a short human-readable name.
Action has a type statement. It defines the job and task clauses.
job is the type of action that starts a single script. Job is specified via the following statements:
-
script -
script_type -
params
task is the type of action that is a chain of jobs. The sequence of jobs to be executed is specified via the scripts parameter. The job sequence is executed according to how it is written in scripts. If any script fails, the execution is stopped and the state of object is set to the value of on_fail.
actions:
install:
type: task
scripts:
-
name: prepare
display_name: Prepare to install
script_type: ansible
script: ansible/prepare.yaml
on_fail: created
-
name: install
display_name: Actual install
script_type: ansible
script: ansible/install.yaml
on_fail: prepare
states:
available:
- created
- prepare
on_success: installed
script is a parameter that takes it value depending on the script_type value.
In case of script_type = ansible, the script parameter denotes the path to the script file.
In case of script_type = internal, the script parameter can take the value from one of the three ADCM functions:
-
bundle_revert— this function returns the state of the cluster as it was before the upgrade. Cluster state includes its configuration, the set of all cluster services, hosts, and components, as well as the host-component mapping. -
hc_apply— this function is useful when action includeshc_acland it becomes necessary to commit the host-component mapping changes regardless of the action execution result or after the certain subjob execution. Thehc_applyfunction commits the host-component mapping changes in accordance with data received from the user. You can use the optionalparamsparameter when it is necessary to commit partial changes. In other words, withparamsyou can specify a rule similarly tohc_acl(the service and component names, and also theaddorremoveoperation), that will commit part of the host-component mapping associated with the specified rule.- name: apply hc script_type: internal script: hc_apply params: rules: - service: redis component: server action: add - service: redis component: server action: remove -
config_apply— this function is useful when action includeswizard_templateand it becomes necessary to save changes in configuration parameter values, for example, according to the data received from a user. You need to specify the rule of committing configuration parameter values and objects using theparamsparameter which describes the object, the service and component names corresponding to the object type, and also the set of key/value pairs.NOTETheconfig_applyfunction is available from version 2.8.0.- name: save display name: "Save configuration on target objects" script_type: internal script: config_apply params: changes: - object: type: cluster parameters: - key: "ssl_default_config/keystore_password" value: "my_secrete_value" - object: type: service service_name: "hive" parameters: - key: "core-site_xml/keystore_password" value: "my_secrete_value" - object: type: component service_name: "hive" component_name: "server" parameters: - key: "core-site_xml/keystore_password" value: "my_value"- name: state_2 display name: "State 2" script_type: internal script: config_apply params: changes: - object: type: cluster parameters: - key: "{{ ssl_key }}" value: "{{ action.process.stages.manage_ssl_stage.configure_ssl.config.ssl_config }}" - object: type: service service_name: "{{ roles_generic_args.service_name }}" parameters: - key: "{{ ssl_key }}" value: "{{ action.process.stages.manage_ssl_stage.configure_ssl.config.ssl_config }}" - object: type: component service_name: "{{ roles_generic_args.service_name }}" component_name: "{{ roles_generic_args.component_name }}" parameters: - key: "{{ ssl_key }}" value: "{{ action.process.stages.manage_ssl_stage.configure_ssl.config.ssl_config }}" -
bundle_switch— this function upgrades all prototypes to their new versions. The function can be used only forupgrade, it cannot be used in the common actions.
script is a mandatory parameter if type = job.
script_type is a parameter that denotes the type of the script, either ansible or internal.
script_type is a mandatory parameter if type = job.
scripts is a parameter that denotes the sequence of jobs to be executed.
scripts is a mandatory parameter if type = task.
You can also pass any additional parameter to the Ansible call. They will be used in ansible-playbook calls.
One special case is parameter ansible_tags. That parameter turns to be --tags parameter of an ansible-playbook call.
You can also pass the jinja2_native parameter. This parameter will be written to the ansible.cfg file. This option preserves variable types during template operations.
Action might have a states optional statement. It defines three clauses:
-
available— list of states of an object for which action will be available; -
on_success— state in which object will be set in case of actions success; -
on_fail— state in which object will be set in case of actions fail.
The on_success and on_fail states are optional. If they are missed, the state of an object is not changed after actions success or/and fail, respectively.
The states statement also has two reserved values:
-
created— the primary state in which objects are immediately after creation; -
upgrading— the state to which the cluster can be put into. In this state, actions such as removing a host or service from the cluster are not available to the user.
masking is an alternative way to describe the action availability in UI and API, which allows you to describe a set of basic (states) and extended (multi_state) states.
Use the masking parameter if you need to describe the action availability using extended states specified in the multi_state parameter.
|
IMPORTANT
DSL masking is not compatible with the states DSL provided above.
|
---
actions:
install:
display_name: "Install Monitor Server"
type: job
masking:
# Action will be shown if both condition under state and multi_state are met.
state:
available:
# If the state equal to any of this, then condition is true
- "state_value1"
- "state_value2"
unavailable:
# If you place unavailable, then no available should be there
- "state_value1"
- "state_value2"
multi_state:
available:
# If we have any of this multistate, then condition is true
- "multi_state_1"
- "multi_state_2"
unavailable:
# If you place unavailable, then no available should be there
- "multi_state_3"
- "multi_state_4"
on_fail:
state: "new_sate_value"
multi_state:
set:
- "multi_state3"
unset:
- "multi_state4"
on_success:
state: "new_sate_value"
multi_state:
set:
- "multi_state3"
unset:
- "multi_state4"
To simplify the rule description, you could use the any scalar that selects all possible values.
---
masking:
state:
available: "any"
unavailable: "any"
multi_state:
available: "any"
unavailable: "any"
If the masking state is omitted, then the action is considered available on every state or multi_state. Hence, the following variants are equal:
---
masking:
---
masking:
state:
---
masking:
state:
available: "any"
The following variants are equal too:
---
masking:
---
masking:
multi_state:
---
masking:
multi_state:
available: "any"
Action might have a config optional statement. It can define a list of values as in config parameters. When the action is run via UI, all those values will be prompted by the user before the action starts.
You can later refer those values in the Ansible script:
{{ job.config.quorum }}
The config statement works with static configuration parameters that are displayed the same way every time an action is started. If you want to work with dynamic configuration parameters, use the config_jinja statement.
An action might have a scripts_jinja optional statement. This statement points to the file of the Jinja format (ADCM uses the 2.xx version).
The scripts_jinja statement is designed for dynamic formation of a set of subjobs based on the context depending on the conditions, in other words, on the cluster topology (for example, whether a particular service was added to the cluster or not). After the action started, a file is rendered in the Jinja format (Jinja2 template) to generate a sequence of subjobs. During the template rendering, the context is formed according to the inventory file, except config group changes, as well as variables and facts that Ansible generates during a playbook execution (including additional action details, and also what was entered and selected by the user when configuring the action).
The scripts_jinja statement is only supported for actions of the task type and is mutually exclusive with the scripts parameter (static subjob generation).
---
actions:
- name: Add/remove components
type: task
scripts_jinja: scripts_jinja/manage_component_scripts.j2
The manage_component_scripts.j2 file:
{%- if (groups['zookeeper.SERVER.add'] | length) > 0 %}
- name: zookeeper
display_name: "Zookeeper"
script_type: ansible
script: ansible/playbooks/cluster/zookeeper_install.yaml
params:
ansible_tags: repo_add, install, configure, configure_main_info, start, bootstrap
{%- endif %}
{%- if (groups['zookeeper.SERVER.remove'] | length) > 0 %}
- name: zookeeper
display_name: "Zookeeper"
script_type: ansible
script: ansible/playbooks/cluster/zookeeper_remove.yaml
params:
ansible_tags: stop, remove
{%- endif %}
- name: zookeeper
display_name: "Zookeeper"
script_type: ansible
script: ansible/playbooks/cluster/zookeeper_install.yaml
params:
ansible_tags: configure
An action might have a config_jinja optional statement. This statement points to the file of the Jinja format (supported in ADCM 2.xx).
The config_jinja statement is designed to work with dynamic configuration parameters. Here "dynamism" means the ability to display (or hide) a certain set of parameters depending on the conditions, in other words, on the cluster topology (for example, whether a particular service was added to the cluster or not). When metadata on action is obtained (via the GET request in the API), a file is rendered in the Jinja format (Jinja2 template). During the template rendering, the context is formed according to the inventory file, except config group changes, as well as variables and facts that Ansible generates during a playbook execution.
The config_jinja statement is mutually exclusive with the config statement which is intended to work with static configuration parameters.
---
actions:
- name: Restart and Reboot
type: job
script_type: ansible
script: ansible/restart.yaml
config_jinja: config_jinja/restart.j2
The restart.j2 file:
- reboot_servers:
type: boolean
default: false
display_name: Reboot servers
{% if cluster.state == 'created' %}
- configure_etc:
type: boolean
default: false
display_name: Configure /etc/hosts
{% endif %}
Action might have a log_files optional statement. It defines the list of log tags. Currently, a check log tag is supported, and you can use the adcm_check Ansible module with it.
Action might have an allow_to_terminate optional statement. This is a Boolean value. If the value is not explicitly set, then the default value is false. If allow_to_terminate is true, then it is possible to cancel the running Ansible playbook and stop the action, otherwise it is not possible.
The allow_to_terminate statement is also applied to all jobs the action consists of. When one of such jobs is stopped, the action continues to run, starting with the job that follows the stopped one. Essentially, allow_to_terminate being specified for an action means allow_to_terminate being specified for all jobs of that action.
Action might have an allow_in_maintenance_mode optional statement. This is a Boolean value. If the value is not explicitly set, then the default value is false. If allow_in_maintenance_mode is true and allow_maintenance_mode in the prototype is also true, then it is possible to run an action even in case of object or just one of its hosts being in maintenance mode.
Action might have the allow_for_action_host_group optional statement. This is a Boolean value. If the value is not explicitly set, then the default value is false. If allow_for_action_host_group is true, then it is possible to launch an action on action host groups.
actions:
upgrade:
type: job
script_type: ansible
script: ansible/site.yaml
allow_for_action_host_group: true
Limitations of using the allow_for_action_host_group parameter:
-
Composite actions (for example, the Upgrade action) are not allowed to launch on action host groups.
-
The simultaneous use of this parameter and the
host_actionparameter is not supported.
This block allows you to tweak how UI displays and performs an action. When you hover the cursor over an action in the UI, the popup message specified in the disclaimer field is displayed.
actions:
install:
type: job
script_type: ansible
script: ansible/site.yaml
ui_options:
disclaimer: "Enter your disclaimer"
An action might have a host_action optional statement which is a Boolean value. The default value is false. If the value is true, then the action appears in the drop-down list of actions on the Clusters → <Cluster name> → Hosts → <Host name> → Host-Components → <Component name> page. In this case, the action is performed on the selected host and a target group that includes the one selected host is created.
It is advisable to declare action (host_action: true) only in the context of a component, since such actions allow you to manage specific components on specific hosts.
actions:
install:
type: job
script_type: ansible
script: action.yaml
host_action: true
Action might have a hc_acl optional statement. If hc_acl is present when an action is run via UI, then a new host-component map will be requested from the user before the action starts.
actions:
expand:
type: job
script_type: ansible
script: ansible/site.yaml
states:
available: all
hc_acl:
-
service: hadoop
component: datanode
action: add
-
service: hadoop
component: server
action: remove
hc_acl defines the list of three clauses that specify operations to modify the component distribution among hosts:
-
service— name of a service to work with. -
component— name of a component to work with. -
action— operation type. Can beaddorremove.
There is a way to specify the Ansible version for the ansible script type actions via the venv optional statement.
Valid values:
-
default— launch action in Ansible 2.8 environment; -
2.9— launch action in Ansible 2.9 environment; -
2.16— launch action in Ansible 2.16 environment (starting with ADCM 2.8.0).
The default value is default.
|
NOTE
With the release of ADCM 2.8.0, Ansible 2.8 is deprecated and will be removed from ADCM in Q3 2026.
|
Change the default environment for all actions on an object:
---
- type: cluster # service component hostprovider host
name: control
version: 1
venv: "2.9"
You also have a way to tweak venv for any actions:
actions:
install:
type: job
script_type: ansible
script: action.yaml
venv: "2.9"
Action might have the wizard_template optional statement which is the template of the flow formed as a result of its rendering.
Unlike the config statement, the wizard_template statement allows you to do the following:
-
divide an action configuration into several logical steps as modal windows;
-
pass the template rendering context between steps;
-
save the states of the steps that are already provided by the user.
The wizard_template statement is mutually exclusive with the following statements:
-
config, which is intended to work with static configuration parameters; -
config_jinja, which is intended to work with dynamic configuration parameters; -
hc_acl, which is intended to commit the host-component mapping changes.
The specified statement is displayed in the ADCM web interface as a wizard which is the preparation phase of an action before its launch.
The flow is a sequence of the following elements:
-
Stages — the containers of logically related steps.
-
Steps — the minimal indivisible elements of a process performed by the user.
NOTEThe process occurs when a user clicks the action in the drop-down list of actions available to work with an object.A step can be one of the two types:
-
Configuration — a configuration filled in by the user. The configuration is dynamically generated during the rendering of the template specified in the
config_templatestatement when receiving information about a specific step. -
Operation — a job representing a set of subjobs that is launched when executing the step. The set of subjobs is dynamically generated during rendering of the template specified in the
scripts_templatestatement when executing the specific step. The example of a step of the operation type is given in the manage_install.j2 file (thecheck_kerberosstep of themanage_kerberos_stagestage).Steps of the operation type provide the following options:
-
validating the action configuration parameter values entered by the user;
-
saving the action configuration parameter values entered by the user in configuration of the target object (cluster, service, or component).
-
-
Each stage includes the following parameters:
-
name— unique stage name; -
display_name— name of the stage that is displayed in the UI for the user; -
steps— sequence of steps.The following attributes are used to describe the step:
-
common attributes:
-
name— step name that is unique to this stage; -
display_name— wizard step name which is intended for use as the name of the modal window step;
-
-
attribute for steps of the configuration type:
config_template— template for working with dynamic configuration parameters; -
Attributes for steps of the operation type:
-
ui_options— attributes specific to UI, in particular,button_name— button name; -
scripts_template— template for dynamic generation of the subjob sequence.
-
-
Template description format (the <wizard|config|scripts>_template statement) includes the following parameters:
-
file— file that is used as a template and contains the following attributes:-
path— path to the file with the .j2 or .py extension. -
entrypoint— name of thecallable()function that should be called (without specifying the package or module). This attribute is used only ifengineis set topython.
-
-
engine— file processing engine. Possible values:jinja2,python.
Based on the received information about an action (if wizard_template is used), a process is created.
actions:
install:
display_name: "Install"
type: task
allow_to_terminate: true
scripts_jinja: scripts_jinja/cluster/manage_install.j2
wizard_template:
file:
path: wizard_jinja/manage_install.j2
engine:
type: jinja2
states:
available:
- created
- faulty_installed
Creating a process, among other things, includes rendering of the file specified in wizard_template. This results in a flow. During the template rendering, the context is formed according to the inventory file, except config group changes, as well as variables and facts that Ansible generates during a playbook execution.
The manage_install.j2 file:
- name: manage_ssl_stage
display_name: "Manage SSL"
steps:
- name: configure_ssl
display_name: Configure SSL
config_template:
file:
path: configs_jinja/manage_ssl.j2
engine:
type: jinja2
- name: manage_kerberos_stage
display_name: "Manage Kerberos"
steps:
- name: configure_kerberos
display_name: "Kerberos configuration"
config_template:
file:
path: configs_jinja/manage_kerberos.j2
engine:
type: jinja2
- name: check_kerberos
display_name: "Check configuration"
ui_options:
button_name: Check
scripts_template:
file:
path: scripts_jinja/cluster/save_config.j2
engine:
type: jinja2
{% if vars.services.hdfs is defined and vars.services.hdfs.state == 'created' %}
- name: hdfs_stage
display_name: "Manage HDFS"
steps:
- name: configure_hdfs
display_name: Configure HDFS
config_template:
file:
path: configs_jinja/manage_hdfs.j2
engine:
type: jinja2
{% endif %}
- name: save_stage
display_name: "Save configuration"
steps:
- name: save_configuration
display_name: "Check configuration"
ui_options:
button_name: Check
scripts_template:
file:
path: scripts_python/cluster/check.py
entrypoint: generate_scripts
engine:
type: python
The manage_ssl.j2 file:
- name: ssl_config
display_name: "Enable SSL"
type: group
{% if vars.cluster.config.ssl_config.enable_ssl %}
activatable: true
{% else %}
activatable: false
{% endif %}
subs:
- name: keystore_path
display_name: "Keystore path"
description: "Keystore path"
type: string
default: "{{ cluster.config.ssl_default_config.keystore_path }}"
- name: keystore_password
display_name: "Keystore password"
description: "Keystore password"
type: password
The manage_kerberos.j2 file:
- name: kerberos_config
display_name: "Enable Kerberos"
type: group
{% if vars.cluster.config.kerberos_config.enable_kerberos %}
activatable: true
{% else %}
activatable: false
{% endif %}
subs:
- name: keystore_password
display_name: "Keystore password"
description: "Keystore password"
type: password
default: {{ action.process.stages.manage_ssl_stage.configure_ssl.config.ssl_config.keystore_password }}
The save_config.j2 file:
{% set ssl_key = "ssl_default_config/keystore_password" %}
- name: validate
display name: "Precheck"
script_type: ansible
script: some/playbook.yaml
params:
ansible_tags:
- check
- name: state_2
display name: "State 2"
script_type: internal
script: config_apply
params:
- object:
type: cluster
parameters:
{% if action.process.stages.manage_ssl_stage.configure_ssl.config.ssl_config.enable_ssl %}
- key: "{{ ssl_key }}"
value: "{{ action.process.stages.manage_ssl_stage.configure_ssl.config.ssl_config }}"
{% endif %}
{% if action.wizard.manage_kerberos_wizard.config.kerberos_config.enable_kerberos %}
- key: "kerberos_default_config/keystore_password}"
value: "{{ action.process.stages.manage_kerberos_stage.configure_kerberos.config.kerberos_config.keystore_password }}"
{% endif %}
The manage_hdfs.j2 file:
- name: kerberos_config
display_name: "DFS data dir"
type: string
default: {{ vars.services.hdfs.config.data_dir }}
The check.py file:
def create_script_template(display_name, name, params, script, script_type):
"""Template for creating action dictionaries with consistent structure."""
return {
'display name': display_name,
'name': name,
'params': params,
'script': script,
'script_type': script_type
}
def generate_scripts(context: dict):
cluster = context['cluster']
action = context['action']
script1 = create_script_template(
display_name='Check',
name='validate',
params={'ansible_tags': ['check']},
script='some/playbook.yaml',
script_type='ansible'
)
script2_params = [{
'object': {'type': 'cluster'},
'parameters': [{
'key': 'ssl_config',
'value': action['process']['manage_ssl_stage']['configure_ssl']['config']['ssl_config']
}]
}]
script2 = create_script_template(
display_name='State 2',
name='state_2',
params=script2_params,
script='config_apply',
script_type='internal'
)
return [script1, script2]
### Return
[{'display name': 'Check',
'name': 'validate',
'params': {'ansible_tags': ['check']},
'script': 'some/playbook.yaml',
'script_type': 'ansible'},
{'display name': 'State 2',
'name': 'state_2',
'params': [{'object': {'type': 'cluster'},
'parameters': [{'key': 'ssl_config', 'value': 'some_value'}]}],
'script': 'config_apply',
'script_type': 'internal'}]
adcm_min_version
Any cluster or hostprovider prototype might have an optional adcm_min_version property. This field defines the minimal version of ADCM that is required for this bundle to work correctly. If the installed ADCM instance version is less than adcm_min_version (defined in a bundle), then uploading of such bundle will cause an error.
allow_maintenance_mode
Every cluster prototype might have an optional allow_maintenance_mode property. This option enables support of the maintenance mode for all prototypes in a bundle. The default value is false.
---
- type: cluster
name: control
version: 1
adcm_min_version: 2019.05.01.00
allow_maintenance_mode: true
components
Service prototype might have components. Component itself is a prototype, so a component description also can contain config or action sections.
components is a way of placing service components on hosts in a cluster.
---
- type: service
name: server
version: 1
components:
master:
display_name: "Master Node"
description: "This node control all data nodes (see below)"
constraint: [1,2]
node:
display_name: "Data Node"
constraint: [+]
client:
monitoring: passive
Every component might have an optional constraint property. It describes how many instances of this component should be installed in one cluster:
-
[1]— exactly one component should be installed. -
[0,1]— one or zero components should be installed. -
[1,2]— one or two components should be installed. -
[0,+]— zero or any more components should be installed (default value). -
[1,odd]— one or more components should be installed; the total amount should be odd. -
[0,odd]— zero or more components should be installed; if more than zero, the total amount should be odd. -
[odd]— same as[1,odd]. -
[1,+]— one or more components should be installed. -
[+]— component should be installed on all hosts of a cluster.
Every service/component might have an optional monitoring property. It defines if the service/component should be monitored. The default value is active. If you want to exclude a service/component from the monitoring system, set it to passive.
---
- type: service
name: server
version: 1
components:
client:
monitoring: passive
Components might have an optional requires property that describes dependencies between the components, i.e. which components need other components or services to be installed simultaneously.
Each component might require one or more components in the same service or in another service of this cluster. Also, a component may need one or more services. If a component requires another component that is absent from a host-component map, you cannot add this component to a host-component map as well. In some cases, all required components should be installed simultaneously. Dependencies defined at the component level affect the validation of the host-component map.
Dependencies can be viewed as another type of constraints (or metaconstraints) for a component.
The example below considers the Hive service with three components: HiveServer, Metastore, and TezUI. The HiveServer component needs the HDFS service, while the information about the HDFS service components is not required. The TezUI component needs the TimeLineServer component of the YARN service.
---
- type: service
name: Hive
version: 1.0
components:
HiveServer:
requires:
- service: HDFS
Metastore:
constraint: [0,1]
TezUI:
requires:
- service: YARN
component: TimeLineServer
constraint: [0,1]
|
IMPORTANT
The bound_to property is not supported since the 2.0.0 version.
|
Components might have an optional bound_to property. Those components should be located on the same hosts as the components they are bound to.
For example, the server component of the pxf service should be installed only on all those hosts where the segment components of the gpdb service are installed. It means the server component requires the segment component and cannot be installed separately.
Hence bound_to is yet another type of constraints (or metaconstraints) for a component.
---
- type: service
name: GPDB
version: 1.0
components:
Segment:
- type: service
name: PXF
version: 2.0
components:
Server:
bound_to:
service: GPDB
component: Segment
config
Config is a general description of object configuration parameters. Configuration parameters can be combined into groups.
There is a number of user-editable values that are related to every object and can be represented in UI.
---
- type: cluster
name: control
version: 1
description: "Monitoring and Control Software"
config:
- name: repos
type: group
subs:
- name: ads
display_name: "ADS repository URL"
type: string
default: https://ci.arenadata.io/artifactory/list/ads-centos7-x64-ads/
required: no
In the above example you can see the ads parameter in the repos group. The ads parameter is not required. This means that it is possible for the user to leave it empty.
Group is not mandatory, so you can place your parameter at the highest level:
---
- type: cluster
name: control
version: 1
description: "Monitoring and Control Software"
config:
- name: ads
type: string
default: https://ci.arenadata.io/artifactory/list/ads-centos7-x64-ads/
required: no
Group is just a visual representation in the UI.
| Name | Mandatory | Values | Default | Description |
|---|---|---|---|---|
type |
Yes |
See below |
— |
Type of configuration parameter. For more information, see the separate Types of configuration parameters section below |
display_name |
No |
User defined |
— |
Readable name that is displayed in UI |
description |
No |
User defined |
— |
Allows you to give the parameter a description |
default |
No |
User defined |
— |
Allows you to set a default value |
required |
No |
yes/no |
Yes |
If set to |
read_only |
No |
See below |
No |
Makes a parameter read-only (protected from change) under certain conditions |
writable |
No |
See below |
Yes |
Makes a parameter writable (available for changes) under certain conditions |
ui_options |
No |
See below |
No |
Allows you to adjust the visual representation of a parameter in UI |
group_customization |
No |
yes/no |
No |
Makes a parameter available to be changed via config groups |
pattern |
No |
User defined |
— |
Validation rule for the entered configuration parameter value |
ansible_options |
No |
See below |
No |
Allows you to specify additional settings for a configuration parameter when preparing Ansible inventory |
The ui_options parameter allows you to tweak how UI depicts config.
The following options are available:
-
Invisible
Add
invisible: trueto your config to hide the config from UI.This option does not hide information from API and does not disable the
read_only/writablelogic. This is simply a UI tweak.int: type: integer default: 1 required: no ui_options: invisible: true -
Advanced
There is a new checkbox that allows you to hide some parameters via labeling them
advanced.Parameter labeled as
advancedis not searchable.float: type: float default: 1.0 required: no ui_options: advanced: true
The group_customization parameter allows you to customize a parameter at the configuration group level.
---
- name: ads
type: string
group_customization: true/false
The read_only parameter allows you to make any config parameter read-only (protected from change by the user or API) for the specified cluster/host/service states. The writable parameter removes the write protection.
The quorum parameter is read-only if a service state is installed or running:
config:
quorum:
type: integer
read_only: [installed, running]
The quorum parameter is read-only for all service states except created:
config:
quorum:
type: integer
writable: [created]
The quorum parameter is read-only for all service states:
config:
quorum:
type: integer
default: 4
read_only: any
With the pattern parameter, you can set a validation rule for the configuration parameter value according to a regular expression. Parameter is supported for the config parameters of the password, string, text, and secrettext types.
The value of the pattern parameter must be a string that is a valid regular expression according to the ECMA 262 regular expression dialect. The rules are outlined below:
-
A single Unicode character (other than the special characters below) matches itself.
-
.— matches any character except line break characters. -
^— matches only at the beginning of the string. -
$— matches only at the end of the string. -
(…)— groups a series of regular expressions into a single regular expression. -
|— matches either the regular expression preceding or following the|symbol. -
[abc]— matches any of the characters inside the square brackets. -
[a-z]— matches the range of characters. -
[^abc]— matches any character not listed. -
[^a-z]— matches any character outside of the range. -
+— matches one or more repetitions of the preceding regular expression. -
*— matches zero or more repetitions of the preceding regular expression. -
?— matches zero or one repetitions of the preceding regular expression. -
+?,*?,??— the*,+, and?qualifiers are all greedy. They match as much text as possible. Sometimes this behavior can be unwanted and you may want to match as few characters as possible. -
(?!x),(?=x)— negative and positive lookahead. -
{x}— matches exactlyxoccurrences of the preceding regular expression. -
{x,y}— matches at leastxand at mostyoccurrences of the preceding regular expression. -
{x,}— matchesxoccurrences or more of the preceding regular expression. -
{x,y}?,{x,}?— non-greedy versions of the above expressions.
Use only standard escapes like \n, \r, \t and notice that you also need to do JSON escaping.
When the pattern parameter is added, consider the upgrade procedure for the current bundle for an existing cluster:
-
Check if the config parameter value matches the
patternand fail the upgrade if it does not match. -
Check if the config parameter value matches the
patternand migrate its value if necessary. -
Check if the config parameter value matches the
patternand raise a flag to notify the user to change the config parameter themselves.
Additionally, for config parameters where the pattern parameter is present, it is recommended to reword the description parameter. You can add examples of characters the value can consist of.
config:
- name: example
type: password
display_name: "Example password"
description: "Use uppercase and lowercase letters, as well as numbers"
default: "qwerty56"
pattern: "[a-z][A-Z][0-9]*?"
The ansible_options parameter allows you to specify options for setting a configuration parameter when preparing Ansible inventory.
The supported option is unsafe which marks the variable value as unsafe to prevent unsafe character substitution in accordance with the Ansible functionality.
---
- name: adb
type: string
display_name: "Name of default database"
default: {%adb
ansible_options:
unsafe: True
-
boolean— boolean value:TrueorFalse. -
file— the content of this property is stored as a file on a file system. When you reference this variable in an Ansible script, it returns a full path to that file.If the default value is set, it should be the path to the file relative to the bundle root directory.
User can edit the content of this file via UI. The file is updated every time after config is saved.
It can be used to store SSH private keys for an Ansible script:
- type: host name: ssh version: 1 config: ansible_private_key_file: type: file default: host/ssh.keyYou can use the special dot syntax to specify a path to the default file. In this case, the default file is expected to be in the same directory as the config.yaml file:
config: ansible_private_key_file: type: file default: ./ssh.key -
float— floating-point number. If the default value is not explicitly set, then it will be0.0. You can use the optionalminandmaxparameters to limit the values:config: average_load: type: float default: 1.0 min: 0.5 max: 3.5 -
integer— integer number. If the default value is not explicitly set, then it will be0. You can use the optionalminandmaxparameters to limit the values:config: memory_size: type: integer default: 16 min: 2 max: 64 -
json— this type stores arbitrary JSON data. The JSON structure is not preserved and can be modified during every config update:config: population: type: json default: - country: Russia cities: - name: Moscow population: 9 - country: USA cities: - name: LA population: 5 -
list— dynamic array of arbitrary length. Array elements should be strings. Any element can be added, deleted, or modified on every config update:config: mount_points: type: list default: - /dev/rdisk0s1 - /dev/rdisk0s2 - /dev/rdisk0s3 -
map— dynamic key/value dictionary of arbitrary length. Keys and values should be strings. Any key or value can be added, deleted, or modified on every config update:config: person: type: map default: name: Joe age: "24" sex: m -
option— key/value dictionary type. The keys are showed in UI while the corresponding values are stored in DB:config: protocol: type: option option: {http: 80, https: 443} default: 80 -
password— same as thestringtype, yet displayed as a password field in UI. -
secretfile— obfuscates parameter and its value in UI. -
secretmap— obfuscates the value that parameter takes in UI. -
string— arbitrary string. The size is not limited. If the default value is not explicitly set, then it will be an empty string"". -
structure— this type is intended for describing multi-level configurations. The number of levels is unlimited. Configurations are described according to the specification outlined below. Upon each configuration update, new data is compared against the specification. If the input data does not match the specification, ADCM issues an error.The root configuration element, which should be the first one described in the specification, is named
root. For further attribute description, thematchelement is used, which can be equated to a data type. Possible values formatchat this level includedictandlist. Depending on thematchvalue, attributes of the next level are added —items(fordict) oritem(forlist). Then, the composition of elements included initemoritemsis listed.You can see available attributes in the table below.
Name Type Required Description match
string
Yes
Rule according to which every other rule performs its checks
item
string
Yes, for
listtypeRule according to which every object is checked in case of
listtypeitems
key/value pairs
Yes, for
dicttypeRule according to which every object is checked in case of
dicttyperequired_items
list
No
List of mandatory object elements. Only for
dicttypeinvisible_items
list
No
Contains elements that can be used for operational purposes and cannot be seen by the user
default_item
string
No
Rule according to which the user-created element in the object is additionally checked. Only for
dicttypeYou can view the available
matchvalues (element types) in the table below.Type Description string
Simple type. Designed for strings
boolean
Simple type. Designed for boolean operations
integer
Simple type. Designed for integers
float
Simple type. Designed for floats
list
Recursive type. Checks itself first according to
item, then applies rules to all its elementsdict
Recursive type. Checks itself first, then applies rules to its values according to
default_itemandrequired_itemsBelow is a configuration example.
config: - name: country_codes: type: structure yspec: codes/schema.yaml default: - country: Greece code: 30 - country: France code: 33 - country: Spain code: 34 regions: - Andalusia - Andalusia - Cantabria - Catalonia - ValenciaThe schema.yaml specification file:
--- root: match: list item: country_code country_code: match: dict items: country: some_string code: some_integer regions: list_of_string comment: some_string uid: some_string required_items: - code - country invisible_items: - uid list_of_string: match: list item: some_string some_string: match: string some_integer: match: intYou can use the dot syntax to specify a path to the specification file. In this case, the specification file is expected to be in the same directory as the config.yaml file:
yspec: ./schema.yaml -
text— same asstring, yet is displayed as multiline text in UI. -
variant— a type where the user can select a single value from the list of options in UI. The sources for that can be different. You can get a list of variants from another config parameter (the type of this config parameter should belist):config: - name: mount type: variant source: type: config name: mount_points - name: mount_points type: list default: - /dev/rdisk0s1 - /dev/rdisk0s2 - /dev/rdisk0s3 read_only: any ui_options: invisible: trueAlso, you can get a list of variants from the built-in ADCM functions, such as
cluster_hosts, that returns the list of all hosts in a current cluster:config: - name: cluster_host type: variant source: type: builtin name: host_in_clusterBuilt-in ADCM functions:
-
host_in_cluster— returns the list of all hosts in the current cluster. -
host_not_in_clusters— returns the list of all hosts not included in any cluster. -
service_in_cluster— returns the list of all services installed in the current cluster. -
service_to_add— returns the list of all services not installed in the current cluster. -
host— receives arguments aspredicateandargs, wherepredicateis a name of internal host function andargsare arguments for this function. All internal functions return the list of hosts (the list can be empty):config: - name: vhost type: variant source: type: builtin name: host args: predicate: and args: - predicate: in_service args: service: UBER - predicate: or args: - predicate: in_component args: service: UBER component: UBER_SERVER - predicate: in_cluster args:List of internal host functions:
-
in_cluster— returns the list of all hosts in the current cluster; -
in_service— returns the list of all hosts in the specified service; -
not_in_service— returns the list of all hosts that are not in the specified service; -
in_component— returns the list of all hosts in the specified component; -
not_in_component— returns the list of all hosts that are not in the specified component; -
in_hc— returns the list of all hosts in a host-component map of this cluster; -
not_in_hc— returns the list of all hosts that are not in a host-component map of this cluster; -
and— returns the union of sets of its arguments; -
or— returns the intersection of sets of its arguments.You can list the variants simply in a bundle:
config: - name: city type: variant source: type: inline strict: False value: - Moscow - Voronez - Ufa default: SamaraYou can also specify an optional argument
strictfor the variant source. Ifstrictis set toFalse, then users can not only select the value from the predefined list, but also enter their own values in UI.
-
-
group— a type that allows you to combine configuration parameters into groups. Group can be activatable. This means that you can switch that group on and off via UI. When a group is switched off (active = false), you do not need to fill in its subvalues in UI. Group value in the Ansible inventory file would be null as well.--- - type: cluster name: Kafka version: 1 config: - name: grafana type: group activatable: true active: false subs: - name: endpoint type: string - name: port type: integer
config_group_customization
Every cluster prototype might have an optional config_group_customization property. This property allows the user to customize all cluster (service, component) parameters on some group of hosts. This functionality is disabled by default. The default value is false.
---
- type: cluster # service component
name: control
version: 1
config_group_customization: true
If you want to change the behavior of a certain parameter, you can use the group_customization option of the corresponding configuration parameter.
description
description is a detailed description of your prototype.
---
- type: cluster
name: control
version: 1
display_name: "Controlling Software"
description: |
This software is intended for monitoring and controlling
you cluster and does it by ...
display_name
display_name is a short human-readable name.
---
- type: cluster
name: control
version: 1
display_name: "Controlling Software"
description: |
This software is intended for monitoring and controlling
you cluster and does it by ...
edition
Every cluster or hostprovider prototype might have an optional edition property that corresponds to the bundle type (cluster or host, respectively).
Default value of edition is community. Value of edition can be used in upgrade definition.
---
- type: cluster
name: adh
display_name: Arenadata Hadoop
version: 1
edition: enterprise
export
Export is a way to specify settings that a cluster or service is able to transfer to other clusters.
If you want to share some parts of your config with other clusters, you can use the export statement. In export you can specify any number of the first level config entries. You can export a config in a cluster or/and in a service:
---
- type: service
name: Hadoop
version: 2.1
config:
core-site:
param1:
type: string
param2:
type: integer
quorum:
type: integer
default: 3
export:
- core-site
- quorum
flag_autogeneration
|
NOTE
Some of this attribute’s functionality was represented by the allow_flags parameter prior to the 2.2.0 release. The allow_flags parameter is no longer supported.
|
This statement is designed for automatic creation of the embedded flags. The statement can be defined on any prototype. The statement can be switched on and off on any prototype as well. If flag_autogeneration is defined both on a child object and its parent object, then the child object attribute is of higher priority.
Currently, only the adcm_outdated_config embedded flag is supported. This flag is raised automatically when object’s configuration is changed. To enable the automatic creation of this embedded flag when the object’s configuration changes, set the enable_outdated_config parameter to True.
---
- type: cluster
name: adpg
flag_autogeneration:
enable_outdated_config: True
import
Import is a way to specify service settings and versions for a cluster or service to be able to receive them from other clusters.
To use the config parts exported from another cluster, use the import statement in your cluster. You should specify the name and acceptable versions of an exported cluster (or a service in a cluster):
---
-
type: cluster
name: Hive
version: 4
import:
Hadoop:
versions:
min: 1.8
max: 2.5
When you bind your cluster and another cluster or service via UI, you can use the exported config parameters in your Ansible scripts like this:
{{ cluster.import.Hadoop.core-site.param2 }}
{{ cluster.import.Hadoop.quorum }}
-
required— theimportstatement might have an optionalrequiredparameter. Ifrequiredis set totruethe cluster or service will raise issue and block any action before the required import is properly binded with export cluster or service. The default value forrequiredisfalse. -
default— theimportstatement might have an optionaldefaultparameter. Thedefaultparameter should be an array of config group names of this cluster or service. If this cluster/service is not binded with any export cluster/service import section, the Ansible inventory file will be filled with values from the specified config group.It’s recommended to make default group names the same as export group names.
-
multibind— import statement might have an optionalmultibindparameter. If it is set totrue, you can bind many instances of a cluster or a service to the same cluster/service. In Ansible inventory file, the import values are represented as an array. The default value ofmultibindisfalse.
license
Every cluster, service, or hostprovider prototype might have an optional license property that corresponds to the bundle type. The value of a license property is a path to the license file relative to a bundle root.
If a prototype of a cluster or a service has a license property, a user of ADCM will be asked to accept a license after the bundle is loaded or the service is added. Before the user accepts a license, any action with the bundle or service is blocked.
---
- type: cluster
name: adh
display_name: Arenadata Hadoop
version: 1
license: misc/license.txt
---
- type: service
name: hdfs
display_name: HDFS
version: 1
license: misc/license.txt
|
NOTE
You can use a special dot syntax to specify a path to a license file. In this case, a license file will be searched in the directory that contains config.yaml:
|
monitoring
Every service or component prototype might have an optional monitoring property. It defines if the service/component should be monitored. Default value is active. If you want to exclude service/component from a monitoring system, set it to passive.
required
Every service prototype might have an optional required property. Default value is false. If cluster cannot work without this service, set it to true. When required service is not added to cluster, cluster will have an issue.
requires
Every service prototype might have an optional requires property.
This parameter defines dependencies between the services, i.e. which services need other services or components to be installed simultaneously.
By default, a service does not depend on any other service or component. However, if a dependent service is specified for a service, then when adding that service without the dependent service, an issue is generated in ADCM. Thus, the dependency of a service on another service is checked at the stage of adding it. The dependency of a service on a component is checked at the stage of the host-component mapping being saved.
---
- type: service
name: Hive
version: 1.0
requires:
- service: HDFS
- service: YARN
component: TimeLineServer
upgrade
Upgrade is an operation available from a particular state of an object (cluster or hostprovider) that updates the metainformation about that object and moves it to another state. There are two types of upgrades:
-
Switching the prototypes of the current bundle to the prototypes of the new one.
-
Performing some kind of action that may consist of several subactions (including switching the prototypes).
Cluster or host prototype might have an upgrade statement. This statement describes how to perform an upgrade from one version of a bundle to another. A prototype might have a few upgrade items to describe upgrades from different versions.
---
- type: cluster
name: control
version: 2.4
description: "Monitoring and Control Software"
upgrade:
-
name: Upgrade 1
versions:
min: 0.4
max: 2.0
description: New cool upgrade
states:
available: any
on_success: upgradable
-
name: Upgrade 2
versions:
min: 1.0
max_strict: 3.0
scripts:
- name: pre check
display_name: Some pre-check before upgrade prototypes
script_type: ansible
script: ansible/pre-check.yaml
- name: bundle upgrade
display_name: Upgrade bundle
script_type: internal
script: bundle_switch
- name: post tasks
display_name: Some operation after prototype upgrades
script_type: ansible
script: ansible/post-task.yaml
from_edition: community
states:
available:
- installed
on_success: upgradable
upgrade should have a versions property. versions is the interval of the versions appropriate for upgrade.
upgrade:
-
name: Upgrade
versions:
min: 0.4
max: 2.0
states:
available: any
on_success: upgradable
|
NOTE
min and max use non-strict comparison (i.e. boundaries include the version of cluster/host). If you want the comparison to be strict, you can use the min_strict and/or the max_strict instead of the min/max.
|
upgrade should have a name property. name is the object name.
upgrade should have a display_name property. display_name is the object name that is shown to user via UI.
upgrade might have an optional description property. description allows you to write description for an upgrade.
upgrade should have a states property. states defines two clauses:
-
available— lists the cluster states that allow upgrade. Special valueanycan be used for upgrade that is available in any state. -
on_success— cluster state after upgrade.
upgrade:
-
name: Upgrade 1
versions:
min: 0.4
max: 2.0
description: New cool upgrade
states:
available: [created, installed]
on_success: upgradable
upgrade might have an optional from_edition property. from_edition is the list of editions that allow upgrades.
Special value any can be used to specify the permission to upgrade from bundles with any edition. The default value of from_edition is community.
upgrade:
-
name: Upgrade 1
versions:
min: 0.4
max: 2.0
description: New cool upgrade
states:
available: [created, installed]
on_success: upgradable
from_edition:
- community
- enterprise
venv
Every cluster, service, component, hostprovider, or host prototype might have an optional venv property.
It defines the Ansible version that will be used when launching an action of the corresponding prototype.
Valid values:
-
default— use Ansible 2.8 when launching an action; -
2.9— use Ansible 2.9 when launching an action; -
2.16— use Ansible 2.16 when launching an action (starting with ADCM 2.8.0).
The default value is default.
|
NOTE
With the release of ADCM 2.8.0, Ansible 2.8 is deprecated and will be removed from ADCM in Q3 2026.
|
---
- type: cluster # service component hostprovider host
name: control
version: 1
venv: "2.9"